Practice English Speaking&Listening with: Building rich fitness experiences with Google Fit platform and Android Wear - Google I/O 2016

Normal
(0)
Difficulty: 0

MUZ MOSTOFI: Hi, everyone.

Thank you.

Thanks for coming.

DAN AMINZADE: My name is Dan Aminzade.

I'm a software engineer at Google,

and I'm also a dedicated triathlete.

So on a good week I'll run 50 miles, I'll bike 100 miles,

and I'll maybe write 3,000 lines of code.

So maybe it's no surprise that I work on the Google Fit

app for Android Wear, which makes

really extensive use of the Google Fit platform

APIs that we're going to be highlighting for you today.

MUZ MOSTOFI: Hi, everyone.

I'm Muz Mostofi.

I'm product manager on the Google Fit platform.

And I'm a keen, quantified surfer.

I enjoy weightlifting, running, yoga,

and occasionally experimenting with different diets.

Today we're going to talk about building

rich experiences with the Google Fit platform and Android Wear.

I'll start out by talking about why

we created the Fit platform, what problems we set up

to solve, and then Dan will go over its APIs and its data

types.

Next, we'll look at some specific examples of what

you can build with a Fit platform,

first basic fitness tracking, and then

a more advanced real time fitness use case.

Finally, we'll finish up with some really cool demos coming

soon.

Give me the clicker.

Sorry.

Let's start out by introducing the Fit platform.

We'll explain why we created Fit, what problems we set out

to solve, and why we believe fitness is such a great use

case for wearables.

Before the Google Fit platform existed life

for a fitness application developer was not easy.

It was integration Hell.

The first issue was that of silo data.

Users health and fitness related sensor data

was siloed across multiple apps and companion devices.

Second, there was no easy access and storage of sensor data.

No easy way for developers to read, interpret, and record

data from health and fitness related devices, sensors,

and connected companion devices such as Android devices.

The third problem was poor interoperability.

There was no simple, standard way for fitness devices

to interoperate.

So in order to integrate with the existing app device

ecosystem app developers would have

to build their own platform and provide an API.

Some of these would be poorly implemented,

leading to user frustration.

With the Google Fit platform we solved that problem

by giving apps one single set of APIs

to access, record, and store phone sensor data,

wearable data, and other app data.

So by using my phone or my wear device as my fitness hub

I can connect my fitness wearable

to my favorite nutrition, and my sleep tracking app

to my favorite meditation app.

We don't just consolidate data, though.

With user consent we provide low level sensor recording,

and we perform hard computations for things

like distance and calories.

Whether it's resting heart rate from heart rate time stamps,

distances fused from your height, steps, and GPS,

or calories from heart rate, the Fit platform

makes it easier for health and fitness application developers.

And fitness and wearables truly go hand in hand.

This is for a few reasons.

First, the sensors.

Your app can get access to all these ways of measuring

fitness activities.

Many watches have a GPS sensor, a heart rate monitor,

and potentially even a barometer.

And because the watch is worn on the wrist

it allows us to detect activities

that we couldn't do with a phone being in a pocket or a purse.

So that's the first reason, sensors.

Second, you can leave the phone.

I love my Nexus 6P, but running with a device

is a bit cumbersome.

I have to rely on infrequent audio queue's to find out

my current pace, because I can't see the screen when it's

in an armband or in my pocket.

Third, faster input.

When you're on the go you can have quick interactions

with a watch to control music or activity recording that

would take much longer if you had

to rely on pulling out your phone

and opening up the right app.

Now let's talk about what the Google Fit platform provides

that enables you to build rich experiences in line

with what we just discussed.

First, let's talk about the Google Fit app.

So here we have the Google Fit app on both phone

and on Android Wear.

If you're not familiar with the app

you can see that it tracks walking, running, and biking

using the Google platform's API that

detect activity, associated steps, calories, and distances.

The app also shows data from connected

to third party devices and apps, such as exercise

sessions, weight, heart rate, nutrition data, and sleep.

Our platform automatically synchronizes

between your Android phone, your Android Wear device,

and the web.

The Fit platform exists in all three of these.

So you can write in data via our rest APIs on the web,

via our Android SDKs on the phone,

or the Android SDK on wear.

The platform handles merging and digi application of this data

intelligently, and we even support take-out integration

if the user wants to export their data.

DAN AMINZADE: Thanks, Muz.

So now that you've gotten an overview of the Google Fit

platform, and maybe some motivation

for why it might be a good fit for your fitness app,

let me go into a little more detail about the different APIs

that it provides.

So basically it breaks down into four APIs.

First, there's the SensorsApi, which gives you

access to raw sensory data streams

from Android devices and wearable devices

sensors, so things like heart rate.

Then there's the RecordingApi.

The RecordingApi does automated storage of fitness data

for you using subscriptions.

You can subscribe to a data type and Google Fit

will store fitness data for that data type in the background,

and it'll persist to the subscription for you.

Next there's the HistoryApi, which

gives you access to the fitness history

and lets you perform bulk operations like reading,

and inserting, and deleting data from the finished history.

And then finally there's the SessionsApi.

Sessions are basically a metadata layer

on top of raw fitness data that let you store whole fitness

activities.

So a session is a time interval during which

the user was doing say a bike ride, or a run,

or weightlifting session.

As far as what you can record we support really

all the standard data types for diet, and nutrition, body,

activity, sleep.

We're also adding this new hydration data type.

I think this is a pretty exhaustive list of fitness

metrics, but if there's one that's

not in here that you'd like to log in your app

you should let us know.

So let's start by talking just about basic fitness tracking.

You can't measure what you don't track so tracking measuring is

really, I think, the first step toward motivating people

towards better fitness.

I actually first started tracking my workouts

10 years ago, back in 2006, when I was

training for my first marathon.

And I bought my very first wrist mounted GPS.

And it was this massive, ugly, uncomfortable thing,

like the size of a chinchilla on my wrist.

But it did the job, you know?

I could plug it into the serial port on my PC

and I could download XML data of my run.

And back then there weren't any nice portals or fitness

platforms so to visualize my runs

I built this website with Google Maps API 1.0.

And I could put my runs up there.

And there's something really thrilling

for a data geek like me to come back from a workout

and analyze all my stats on pace, and heart rate,

and elevation.

And I could really track my progress from week to week,

and workout to workout.

So these days obviously there are much more elegant

and full featured alternatives where

I can analyze my workouts, and share them with friends,

and track my progress toward fitness goals.

And also I don't have to wear a big, giant ugly thing

on my wrist.

I can wear these beautiful time pieces

from TAG Heuer or Michael Kors, and I can still

track and visualize my progress right on my wrist

throughout the day with the watch face.

Now this is actually my favorite watch face here, the Fit Cat.

Yeah.

Big ups for Fit Cat.

It tries to encourage you to be more active.

So, basically, if you haven't seen this before,

the more active that you are, the more steps you take,

the more active the cute, little cat becomes.

I love how he's wearing the I/O badge

and he has the backpack there, and wearing an Android Wear

watch, also.

So this seems like a simple concept, right?

The more steps you take, the more active the cat becomes.

But actually trying to build something like this on your own

is not easy.

There's a lot of stuff you have to do if you

want to do it on your own.

You have to build a data store that tracks steps.

You'd have to turn on sensors, get permission

from the user to do that.

Then you'd have to be able to record

step counts in a way that won't kill the watch battery.

If you get that part working then when the user first

installs your watch face you won't-- it will show zero

because you haven't been recording yet.

And then there's all these edge cases,

like the user takes off their watch

and goes for a walk with just their phone in their pocket,

then you have to write the logic for merging in that data.

And then even if you solve all these problems

your step counts still is probably not

going to be consistent with other watch faces

in other apps.

So, fortunately, we solved all of these problems for you

with the Google Fit platform.

We store steps in the background in a nice, low powered battery

friendly way, and they're ready to be read anytime you like.

And you can get consistent step counts

on Android Wear with just one method call.

So let me just show you how to do that.

Typically when you want to connect to the Google Fit

platform you create a Google API client, right?

And normally you create an authenticated one

where you specify the account that you want to read data for.

So here's how I build the client.

I say that I'm interested in reading fitness history data

so I add the fitness history API.

I add scopes for the particular types of data I want to read.

So in this case activity type, location, and body sensor data,

like heart rate.

And then next I add an account.

So this is the account I want to read the data from.

And then finally I add my callbacks.

And now I have this authenticated client

that I can use to make queries to the fitness platform.

But wait, you're saying, Android Wear watches

don't have accounts.

And this is true.

Android Wear 2.0 will actually add account support,

but for now we don't have accounts on watches.

Fortunately, there is a solution.

Now you could sync account information from the phone,

but this would require a watch face

to have a companion phone app, which

is a lot more complicated.

So here's an easier way to do it without a phone app.

You can create a no account client.

So you create a Google API client.

As before, we're going to add the History API.

And now we just call set default account.

We don't specify an account.

Finish building it, and now we have a new account

client that we can still use to query step counts.

And the way you do that is with the read daily totals method.

So you're going to call the History

APIs read daily total method.

And this is basically going to give you a simple way

to get step counts for watch face that's

going to be consistent with other apps and other watch

faces.

You can set a callback so that when the result returns

you can process it.

So here in the on result I'm basically just extracting out

the step count, putting it into a steps total variable,

and there you go.

It's a pretty simple way to track steps

with your watch face.

And it's not just steps that you can get from the Google Fit

platform.

You can also read and write other data,

like calories consumed.

So here's an example of how the Lifesum app displays that data.

Here I'm looking at the amount of calories

I have left in my budget for the day, 911.

So I guess I could eat like two avocados

and drink two glasses of wine tonight.

Android Wear 2.0 is going to have a new way

to show data on watch faces using complications.

Maybe you've seen some of this earlier in the keynote.

But, basically, on the right there's

a watch face that has three complications.

And the watch face lets you select

which complications you want, and then any app

can provide data for those complications.

So here's an example where Lifesum is exporting data

on calories burned as a complication

so that the ranger watch-- it's switch is

on the right-- can display it in one of those three user

configurable slots.

And if you want more information on building complications

in Android Wear 2.0 highly recommend you check out

the talk-- there's a dedicated talk at 4pm on this topic.

MUZ MOSTOFI: Another challenge that fitness developers

face is how to perform difficult computations.

For example, to derive distance traveled

from multiple sensory inputs the user's height,

stride length, GPS data, activity type, active time,

and step counts all have to be fused in together, potentially

across multiple sensory inputs.

The Fit platform makes this easy for third party developers

through its advanced transformations

for both distance and calories.

The basis app takes advantage of this.

Instead of computing distance themselves

they write in steps and activity data

into the Google Fit platform, and read back

our computed values for distance.

DAN AMINZADE: Cool.

So let's take a look at how we might do something like this

with the Google Fit platform.

So there is really-- there are two ways of writing data

to the platform.

You can use the recording API to do live recording.

And then you can use the History API to insert activity history.

So let's actually start by looking at live recording.

All we have to do to start recording

is to subscribe to the data type we care about.

So here I'm calling subscribe and I'm

saying I'm interested in activity samples--

activity type.

And then-- I actually, at that point, I am essentially done.

Google Fit will start storing that data

that I care about in the background.

So I don't need my own database or my back-end to store it,

there's no listener needed here.

Basically, the Google Fit platform

is going to start recording data and syncing it

to the cloud in the background.

And when I want it later I can just

read it with the History API.

So once I subscribe I'm done.

All I really need to do is just check

the status code in the on result.

And that's just to make sure that the subscription

succeeded.

Now let's look at another way of getting data

into the Fitness history.

Suppose I've gotten my data from some external source,

like another device.

So in the example of basis they wanted to write in some steps

that they had gotten from another device.

So first you create this data source object,

and you specify the type of data you want to write here.

I'm saying I want to write raw step count data.

And then you create a data set from the data source.

And you add one or more data points to the data set.

Each of these points has a start and an end time.

So here I'm adding 950 steps with a start and an end time.

And then, finally, you just call insert data

to put the data into the fitness history, and that's about it.

All you really have to do after that is just basically verify

that the operation succeeded before you start querying

the data with the history API.

And one nice thing is that the Fit platform will actually

validate the data points that you insert.

So, for example, if I put in an invalid time range,

or I put in a really unrealistically high calorie

burn, or cadence, or something like that,

I'll get an error back in the client instead of the user

history getting all messed up.

So here's how this actually looks in the basis app.

They put in steps, and then you can

see they are able to derive from that-- or through the Google

Fit platform they get this accurate distance traveled.

MUZ MOSTOFI: We've been working hard over the past year

to improve the quality of these transformations.

The first area with which we've improved them

is relation to distance.

Distance computed from steps now has been improved

with stride length calibration.

From actively trucked walks and runs

to improve the quality of passively tacked

walks and runs.

Secondly, for calories, we will soon

be leveraging gender, age, and heart rate

as part of our computations to improve the accuracy.

Thirdly, for merging.

Previously leaving your phone at your desk

and going for a walk with your Android Wear device

would result in your step counts dipping back down

and then rubber banding back up.

That will no longer be the case through bi-directional syncing.

We've also made merging of our GPS data

better by prioritizing the highest fidelity GPS sources.

Finally, with regards to activities,

this is our most complex task, to infer the activity being

performed by analyzing multiple sensory inputs in real time.

It's really important, as users get frustrated

when their fitness data has false positive and false

negative activity.

And they don't get their proper calories and step counts

as a result.

Fortunately, Google is good and trusted

at collecting and understanding complex data

through our advanced machine learning capabilities.

We've added dynamic sampling of sensor inputs

to improve quality, and have made improvements

to our algorithms by fusing steps, position, speed,

and user corrections together.

The result of this hard work over the past year

is a reduction in our biking precision error rates

from 11% error in early 2015 down to 4% error as of now.

And these changes will be rolling out now.

Finally, we've added stride length calibration

to the Fit platform.

Which we hope will greatly benefit non-GPS Android Wear

devices.

So now if a user goes running initially

with their phone and their watch, which

has-- the phone which has GPS and the watch,

which doesn't, the two will calibrate together and generate

a stride model and then they'll get accurate distance and pace

estimates running without their phone,

just with their non-GPS Android Wear device.

Now that we've looked at some basic fitness tracking examples

I want to talk about our push this year

into real time fitness tracking.

But why is real time so important

when building a motivating fitness experience?

Well, when I look at fitness apps out there--

and I look at a lot of fitness apps out there--

I see that they adopt this framework.

They start with the goal of the user.

For example, I'd like to lose 10 pounds.

I'd like to sleep better, or I'd like to finish a half marathon.

They recognize that in order to accomplish this goal

they need to encourage a pattern of motivating actions

so that the user achieves the result that they desire.

Let's drill into these motivating actions.

There are three pillars of wellness, exercise, nutrition,

and recovery.

Recovery we mean to mean sleep and meditation.

But these pillars do not operate independently.

And behavior in one needs to be balanced against the other.

By getting a holistic view of what's

going on in each of these areas your app

can more intelligently and insightfully

present suggestions and nudges based on a changing fitness

context in real time.

For example, a sleep app might observe

that you sleep better when you exercise,

or when you eat less sugar in the evening.

An exercise app may suggest exercise

based on your logged nutrition data

so that you meet your weight loss goals.

Note, that in all these cases timeliness is important.

The best time to coach and motivate a user

is immediately in response to an action

that they have taken in relation to their fitness.

So good apps have an inbuilt awareness of the user's

overall fitness context.

One of our partners, Lifesum makes

heavy use of these principles in their application.

And they integrate deeply with the fit platform APIs,

and as a result they've seen double the retention rate

for their users connected to Fit versus their users

who aren't connected to Fit.

And who are the APIs that we've added

to let you respond in real time to a changing fitness

context for your user?

First, there's Real Time Data Update Notifications.

It allows you to be notified for certain data

types in the fitness store when they change.

The second is Real Time Session Notifications.

It's actually been in the platform for awhile,

but we'd like to talk about it today.

It allows you to be notified that the user has started

or finished a workout tracked in the third party application.

And, finally, there's Real Time Activity Recognition.

Coming soon to Android Wear devices

to detect the onset of activity to launch into your activity--

active tracking experience.

Let's dive into the first API, Real Time Data Update

Notifications.

This allows us to build the following experience.

Say I just had an impromptu pizza

and I've exceeded my calorie intake goal for the day.

I can get a notification from my weight scale app telling me,

listen, more exercise is needed to keep you on track

to meet your weight loss goal.

DAN AMINZADE: I do admit I have a weakness for pizza

and that one looked pretty delicious.

So let's see how we could build something like that example

that Muz just described.

What we want to do is subscribe to updates

to the nutrition data type.

So first thing we do is create a PendingIntent.

This is typically a PendingIntent for an Intents

service that's going to call into your app

whenever an update to one of these data types occurs.

What you then do is you create a DataUpdateListen

erRegistrationRequest, which is a lot of camel case,

but it is a pretty descriptive name.

And we're going to pass in the type of data

that we'd like to subscribe to updates for.

So in this case nutrition, we're interested in meals

that are being logged.

And then we pass on that PendingIntent we created.

And then, finally, we just call this

RegisterDataUpdateListenerMethod and we pass

in our client and our request.

And then, voila, we'll get an update

with the next logged meal.

And, of course, you can register for updates

to all different kinds of data types.

For example, if I want to be notified every time

the user steps on their scale and logs a new weight

I can do that with type weight.

And here are actually all of the data types

that support real time updates.

Pretty much anything that isn't updated super frequently.

MUZ MOSTOFI: The second API, Real Time Session Update

Notifications, allows, for example,

a nutrition app to be notified that a user just

finished a workout session.

So say you just finished a weightlifting session,

your nutrition app can tell you, it's time

to get a high protein meal in to maximize your muscle growth.

Or, say a hydration logging app.

It can suggest rehydrating immediately

after finishing a 21 minute run outdoors.

DAN AMINZADE: Yeah, It's been so hot here I don't think--

I think we can all use hydration reminders even

if we haven't been running, just walking around outside.

So let's build that one.

Similar API to the one that we just

looked at with the HistoryApi, but here we're

going to use the SessionsApi because we're

interested in registering for updates

when a session starts or ends.

So again we create a PendingIntent

that we'd like to fire when a new session begins or ends.

And then we call this register for sessions method

in the Sessions API.

And now whenever the user starts or ends a workout

the PendingIntent will fire and intent

will be delivered to our app.

And if you want to get more information on that session

you can actually extract it from out of the Intent.

So here I'm pulling out the start and end time, and then

also the type of workout.

So whether it was a run, or a bike ride,

or a weightlifting session, and I

can put that into the notification like the one

that Muz just showed.

One exciting new development that's

coming to the Google Fit platform this year

is that we're actually going to allow your app

to register not only for these explicit sessions

that the user initiate, but also for implicit sessions.

These are ones that are automatically

detected by our platform.

So what it means is that your app can respond in real time

to user's fitness activity.

So if I just lace up my shoes and I

start running my favorite run tracking app

can just start up automatically on my watch.

So I'm going to try to show you a demo of how that works.

I'm not really properly dressed for running,

but I'm going to do a little run around here.

OK, I just felt my wrist vibrate which

means that Strava should have started up on my watch.

So let's go over here and take a look.

And there we go.

So you can see that Strava is now recording my run.

And I'll post that on Strava later

so I hope you can give me some kudos.

Here you go, Muz.

MUZ MOSTOFI: Awesome.

So we got a couple of really exciting--

well three, actually, really exciting new features

going on Android Wear towards the end of the year.

We've added a Beat to Beat API in Android M. What this allows

you to do is get the heart rate time stamps in addition

to the BPMs.

And compute heart rate variability,

which is a proxy for stress levels for the user.

Second, we're adding a profileApi.

This allows you to easily onboard

you new users by reading age, gender, height, and weight

from the Fit platform.

And, finally, GoalsApi allowing you

not only to show, for example, just the step count,

but also the progress against that step goal.

But also active time, distance, and calorie

goals as well from Google Fit.

There's another exciting feature that some of guys

might have seen at the sandbox already.

so when I go to the gym I notice that a lot of people

are still using an old school pen

and paper to log their exercise, their reps, their weight.

Seems a bit crazy in 2016.

Well Google fit on Android Wear did count reps for you

for three specific exercises, push-ups, sit-ups, and squats,

but only if you told the app in advance what you were doing.

This is still somewhat limited and cumbersome.

Now we've got something better.

So I'm going to show you a demo.

DAN AMINZADE: Oh, come on, Muz.

MUZ MOSTOFI: Oh, all right.

All right.

OK.

Maybe.

Oh.

DAN AMINZADE: Give me a break.

MUZ MOSTOFI: OK, so I'm going to the Google Fit strength

training section.

And I haven't told it the exercise I'm about to do.

DAN AMINZADE: Come on.

Woo!

He's working hard for you guys.

MUZ MOSTOFI: As you can see it detected bicep curls with 100%.

[APPLAUSE]

And counted the reps.

It was 15 pounds so I'll just log that.

Let's do another exercise, one which involves the legs.

Note that I can take some time just kind of setting up.

DAN AMINZADE: Very good form.

MUZ MOSTOFI: That's not.

[LAUGHTER]

OK.

I'll put these down.

And you can see it got deadlift.

DAN AMINZADE: Yay.

[APPLAUSE]

MUZ MOSTOFI: If a rep is missed, or an exercise isn't correct,

for whatever reason, we will learn from that mistake

and become trained to the user's exercise motion.

We aim to seamlessly track a whole workout with minimal user

input.

We're planning an API towards the end of this year

and we're looking for partners.

DAN AMINZADE: All right.

So that's our overview of the Google Fit platform.

Before we wrap up we just want to thank all of our partners

who are already using the Google Fit platform to inspire

and motivate people around the world

to lead more healthy and active lifestyle.

You know, at the end of the day this talk isn't really

about pending intents and broadcast receivers,

it's about improving lives and extending life spans.

And so we really hope that you'll

be inspired to get out there and build

the next generation of awesome, amazing fitness experiences

on Android.

So thanks so much for your time and attention.

Feel free to contact us with any questions.

Thank you.

[APPLAUSE]

[MUSIC PLAYING]

The Description of Building rich fitness experiences with Google Fit platform and Android Wear - Google I/O 2016