Follow US:

Practice English Speaking&Listening with: Innovation at Google

(0)
Difficulty: 0

DOUGLAS MERRILL: Hi, I'm Douglas Merrill.

I want talk today about innovation at Google.

Thank you very much for coming.

I'm really excited to get the chance to talk about this.

We are an engine for innovation.

Yes, technically, we're a search, advertising, and

applications company, what we call search, ads, and apps.

But the reason we've been successful at search, ads, and

apps is because we've managed to innovate and find new and

unusual ways to solve really, really hard problems. And

that's what I want to talk about today.

Google was founded in the late '90s by two graduate students

with this amazing idea to take all of the world's information

and make it universally accessible and useful.

All of the world's information universally

accessible and useful.

The three parts of our mission that operate us today: all the

world's information, not just the web; all of the world's

information, not just the web in English; the web in any

language, books, print, radio, TV, all of the world's

information.

Universally accessible, although I'm projecting this

deck today from a laptop, in lots of the world they don't

have good wireline internet access.

Or laptops are very expensive.

So, to make the information universally accessible, you

might need to go over a telephone, or you might need

to go over some other kind of kiosk device.

Universally accessible, all of the world should get to reach

out to this great new world.

And useful, the hardest part of making information is not

actually making it available, it's making it useful

once you have it.

You can present large tables of numbers that have all the

information value, but if you don't put them on a map, those

numbers are less useful, et cetera.

We spend a lot of our time, and a lot of our AI work, and

a lot of our innovation around figuring out ways to make more

information useful.

Let me talk for a couple of seconds about what search is.

Google is a search company at heart.

We know how to provide search experience.

And there are really four parts of search.

And I want to talk about this, not to make you Google

experts, but rather because it helps provide context for some

of what I'm going to say later.

The first thing, when people talk about search--

you see this in press reports of search a lot--

they ask about how big is your index?

How many documents do you have?

And we don't talk about that a lot because it's not really

the right number.

We talk about comprehensiveness.

Comprehensiveness is more than just index size.

Comprehensiveness is how much of this information of a kind

do you cover?

So web comprehensiveness says how many web

pages do you have?

You could probably count them, but the counting's kind of

meaningless.

And so that's why we don't really engage in it.

Do you count the same page twice.

How do you decide?

The point is, if someone has a query that should be answered

by a web page, do you have that web page in your index?

That's comprehensiveness.

The problem is, lots of queries shouldn't be answered

by a web page.

If you do a query, 'What is Winston Churchill's birthday,'

you ought to get a date back, not just a web page.

If you do a query that says, 'Where is Google

headquarters," you should get a map back, not

just a Google page.

Sometimes comprehensiveness means more

than just web pages.

And, if you think about those examples I just gave you,

relevance is also important.

Relevance is the second component we talk about in

overall search.

Relevance is a term of art, meaning that the results that

appear close to the top are actually what you meant.

So, notice what I just said, "what you meant." Our goal is

to have the top result be not just what you asked for, but

what you should have asked for, what you really meant.

Let me give you an example.

If you do a query, you do a search for the word A- P- P-

L- E- S, the word apples, you're going to get a bunch of

results back, including stuff on fruits and all kinds

different stuff.

If you do a search for A- P- P- L- E, apple, you're going

to get another set of results back.

Mostly about a computer manufacturer because our

automated and objective algorithms have figured out

that users, when they do a search for Apple, are more

often looking for a computer manufacturer--

now a phone manufacturer, or an Mp3 player manufacturer--

than they are for a fruit.

When we talk about relevance, we mean the innovations that

we've done--

innovations, note: foreshadowing a cheap

technique used by authors--

the innovations that we've done to make the results more

likely to be what you wanted not just what you said.

Another interesting thing, speed matters.

I'll talk about this a lot more later.

Latency matters a lot.

In fact, in a minute, I'm going to give you an actual

statistic that shows that we can lose 15% of our traffic

just by slowing down 200 milliseconds.

It turns out speed matters a lot.

Speed matters as much as anything else.

If you have a comprehensive, good, highly-relevant answer,

and it takes too long to respond,

your users will opt-out.

So we've done a whole bunch of technical work to make it

possible for us to answer any question, anywhere in the

world, in about 400 milliseconds.

400 milliseconds?

Not a long time, unlike how long I'm

talking on this slide.

So I should pick up the pace.

The most important part of search is the user experience.

Do your users get what they want, how they understand it?

So recently, we announced One Google, or what we call

universal search, this idea that you can do a search for

Darth Vader and get, not just web pages about Darth Vader,

and maybe some images about Darth Vader, but also, maybe,

some video clips, and maybe some blog content,

and some news even.

Although, there hasn't been news about Darth Vader lately

that I'm aware of.

Maybe there has.

Is there a new Star Wars coming out?

Anyway, That's not relevant right now.

That was a pun to my previous slide here.

You guys are a tough crowd.

OK.

The most important part of the user experience is to make it

possible so that users understand, intuitively, what

they're getting.

That's what makes a good search engine.

Search is a very hard problem.

It's not enough to have a very big index.

An index of what?

You cover the right kinds of documents.

It's not enough to have a big index if you don't reply with

the right kinds of relevant, important, meaningful results.

It's not enough to have a big index and reply with the right

kind of meaningful, important, results if you do it slowly.

And finally, it's not enough to do all those things if you

do in a way that the user can't understand, and you

don't help the user have the experience of what they're

trying to find.

Those are the components of search.

And search is still not solved.

We've been working on it for a long time.

We have persons centuries of innovation into this to get

where we are today.

And yet, there are a lot of problems, that we're still

working on, that are really, really hard.

Google Book Search, for example, a really interesting

project we have going on that's partially with

publishers and partially with libraries to try and

understand the content that's in books.

We're scanning the content and making it so you can search.

And then, you can search for content which is in books.

There's lots of great content in books.

I like books.

Most people have read books.

Have you guys read a book?

Wow.

They're both saying no.

That's scary.

Anyway, the point of a book search is to say there's

information which isn't on the web.

Remember our mission: all of the world's information

universally accessible and useful.

It's useful to be able to ask Google, "Who is Ishmael," and

get an answer back that says you should go buy Moby Dick

from Amazon or from your local book seller.

Access on mobile, I've talked a little bit about that.

I'll talk more about it later.

It turns out that there are millions of cell phones in

Africa with non-wireline internet access, and a handful

of wired internet access points.

If you want all of the world's information to be universally

accessible, you need to figure out the mobile access problem.

But, if you think about it, on a phone, you probably

shouldn't interact the same way you interact on a

keyboard, right?

You know, tri-touch is hard and the screens are very

small, so pages have to be re-rendered.

It's very complicated.

There ought to be something like, maybe, you call a phone

number and you speak into it, and you get

search results back.

Or maybe they text you the search results back so you can

choose between them.

There ought to be some way of recognizing as you move from

one medium to another.

So what that means is you went to Maps.

You found your start address.

You found your end address.

You did a search.

You printed it out, and then you walked off without it.

Why is it that that map just doesn't automatically follow

you to your phone?

It ought to be about you.

Problems like that, search is a fascinating thing across

access mechanisms.

Let's talk, briefly, about machine translation.

One of the very hardest AI problems, ever, is machine

translation: translating automatically from one

language to another language.

Google has recently won a series of awards from the

Association of Computing Machinery for automated

machine translation from English to Arabic and back,

and English to Chinese and back.

What that means is we have machines that analyze an

English stream, or a Chinese stream, and

translates it into the other.

I'm a very big soccer fan.

It turns out the rest of the world

calls that game football.

Americans call something else football.

When I do a search for soccer, it ought to be the case that

Google can return web pages, or news results, or video

results, in Italian about football, using whatever the

Italian word for football is, right?

Machine translation is another way to let people have access

to all of the world's information.

One of the greatest side effects of the internet

revolution has been the democratization of

information: the ability for people to tell their story

regardless of who they are.

In the past, history was always written by the winners.

Pop quiz: what's the difference between a

revolution and a civil war?

Answer: who won?

If the folks having the revolution won, it's a

revolution.

If the government won, it's a civil war.

The only difference is who won.

History is written by the winners.

That's not the right answer.

History ought to get written by everybody.

And one of the great things about the internet, and

companies like Google, and others, is that we've given

tools for everyone to tell their story.

But it's not enough to be able to tell your story, you have

to be able to find that story.

Search is the oxygen of the information economy because,

without search, you can't find information.

If you can find information, you can find multiple

perspectives.

If you can find multiple perspectives, you get to have

different, more innovative, answers.

Google is an innovation engine, and we are applying

those innovations to these problems and others.

Somehow, you all have been nice enough to let me use the

word innovation, probably 1,000 times already, without

defining it.

There are really three kinds of innovation.

Innovation's an interesting topic in the world.

Fundamentally, about every five or six years, there's a

wave. That wave is something like this, somebody publishes

a Forbes paper, or a Harvard Business Review article, or

something, that says, "Wow, companies need to innovate

more, because, if they innovate more,

they'll get new products.

They get new products, they'll get new markets, and they will

own the world.

A new hegemony will arise." And so there's a bunch of

articles about innovation and, often, references to hegemy--

hegenomy--

Agh!

Hegemonic kinds of companies.

I struggled a lot over that word.

I'll keep trying it.

I'll throw it in a few more times into the talk.

You guys can all laugh when I screw it up.

The problem is, that goes on for a while.

There's lots of focus, and lots of focus.

And then, suddenly, somebody writes the

counterpoint article.

The counterpoint article goes something like, "Oh, yeah.

Innovation is not useful.

Innovation just wastes money.

Fundamentally, a company should focus on their core

business because, clearly, innovation is antithetical to

your core."

And then, you're on the downward cycle.

And, all of a sudden, everyone's like, oh, we

shouldn't innovate.

We should execute better.

And there's this move back to scientific management.

And then, two or three years later, we

start it all over again.

So we're, sort of, over here, it seems to me,

on this wave, right?

We have had the recent articles in the popular

business press talking about innovation for its

own sake, et cetera.

The problem with the whole conversation is they also

don't define what innovation means.

The truth is, I agree.

Innovation, for its own sake, isn't useful.

So then, what is useful?

Let's step back.

What is innovation?

There are really, if you think about it, three kinds of

innovation.

There's incremental innovation: a little thing

changes that changes something little, small.

There's a great biological example of this, right?

Evolution.

Sorry, for the creationists in the audience.

Sorry.

Anyway, evolution exists.

OK.

Evolution works on all this big thing of DNA, and makes a

little change.

And most of the changes are irrelevant.

And some of the changes are really bad.

And some of the changes are good.

But each one's little, tiny, incremental innovation.

It's also, in business, was that everything we talked

about in the '70s and early '80s around lean manufacturing

for cars, the idea that we should move the tools slightly

closer to the workers because that saved an extra movement,

and that saves a few minutes for a car; and quality is

free; and incremental innovations, small changes to

process; good to do.

But not the only kind of innovation.

There's also a second order of kind of innovation, what I

call, incremental innovation that has a side effect.

A biological example of a side effect: the opposable thumb.

If you think about it, animals farther down the evolutionary

ladder than us don't have opposable thumbs, they have an

extra finger up there.

And, over time, that finger moves over,

and over, and over.

And, all of a sudden, it gets far enough over to where its

axis of rotation is against the axis of rotation of the

other fingers, and you have an opposable thumb.

What happens the minute you have an opposable thumb?

You have tools.

You have pencils.

Suddenly, the world changes.

It wasn't that big a change, right?

It's just a few degrees of axis of rotation of a joint.

But that side effect creates civilization.

Incremental innovation with side

effects: very, very important.

And then there's the thing that we all think of as

innovation: transformational change.

Transformational change, suddenly something has

changed, and the world changes.

Everything is different.

Biological examples of

transformational change are hard.

You know, maybe it's the asteroid hitting in northern

Siberia causing the extinction of lots of species, maybe.

Or maybe it's the evolutionary change of your gills falling

off because the minute you don't have gills anymore you

better be able to move on dry land, or you're done, right?

Without gills in the water, you're drowning, yeah?

Gills?

Gills?

Fish?

Fish?

Come on.

Yeah, you're awake, right?

OK.

Transformational things don't happen a lot in biology

because they're not very efficient,

they're not very effective.

They happen a fair amount in business.

I would argue that a transformational change in

business is Google's auction model for ads, where the price

for an ad is based on someone clicking on the ad.

And the value of that ad is the economic value created by

that ad clicking, which is actually the price bid by the

one under you.

So I would argue that the concept of targeted internet

advertising is a

transformational change to business.

But still, there aren't a lot of examples of it, and it's

what we all spend all of our time talking about.

Let me give you a few examples, in Google's context,

of all three kinds of these innovations, if I may.

You know, I'm a Google person talking about Google.

Thank you for bearing with me.

If you do a web search for the words "heart attack," I don't

really, actually, know what you're looking for.

If you were to grab a medical textbook, just an average

medical textbook, and open it up to heart attack in the

index, you're going to find a whole page of entries of

different kinds of things.

And each one refers to one or two pages.

So you're at an index entry for heart attack, and this

index has 100 sub-entries.

I don't know what you're trying to do.

So, originally, we noticed the same thing on the web.

People would do a search for heart attack.

And they go to a page.

And they come back up and they do a search for, oh, maybe,

heart attack symptoms, or heart attack treatment.

In fact, we have a story of a person who did a search for

heart attack, and looked at some pages, and then did a

search for heart attack symptoms, and then called 911

because he discovered that his symptoms matched, that he was

having a heart attack.

He went to the hospital and lived.

He was, in fact, having a heart attack.

It's a tremendously heartwarming story.

On the other hand, why make him do two

searches to find that?

We could have saved him, oh, I don't know,

30 seconds, or whatever.

So now, if you do a search for heart attack, what you'll see

is a little box.

And we called it-- down at the bottom---

what we call, search refinement.

These are other things that people who've searched for

heart attack have looked for.

We didn't have some human go through the logs

and figure it out.

We did an automated and objective analysis using

algorithms to study what happens in searches.

And we found that people who are looking for heart attack

are often looking for information for health

professionals, or causes and risk factors.

That's an incremental innovation, which adds a lot

of value to our users.

Incremental innovations are very useful.

We would have saved that guy an entire search.

However, we also have examples of incremental innovations

that include side effects.

So, here's a side effect.

Google has a business whereby we will syndicate ads, run our

ads on other people's content, it's called an AdSense.

In this case, you can see in the bottom, right-hand side of

the screen, we have some ads we're showing, and there's

also a banner ad on the top.

This particular page is about space exploration, so

presumably, someone did a search for space and got led

here, et cetera.

OK, now, that's sort of an interesting article.

And it talks a little about geology, so it's kind of a

good science article.

Our ads are prints of the space shuttle.

OK, reasonable.

If you're interested in space, you might be interested in a

wall hanging and other kinds of things like that.

And also we have a newsy update on the Mars rover.

The random ad is an Army ad.

The connection between space travel and the Army escapes

me, slightly.

The side effect here is that not a lot of people would

click on the Army ad.

That Army ad didn't add a lot of economic value to the

creator of this page.

However, lots of people click on our ads.

When someone clicks on a syndicated ad, if someone has

a content page and they show one of our ads, and a person

visits that page and clicks on an ad, we

get some of the revenue.

And the content creator gets some of the revenue, the

person who wrote that page.

What's the side effect?

The side effect is we made it possible for people to make

their entire living, or part of their living, writing

content online, sharing their stories.

I talk about the democratization of

information, everyone gets to tell their story.

Only rich people get to tell their stories full-time, rich

people or people who are getting paid.

We provided a way for people to get paid to tell their

stories in an effective way.

A side effect of this innovation was the creation of

a more democratized information field.

Really great side effect, so was the opposable thumb.

We also have transformational innovations that

have happened at Google.

Google started as a random assortment of hardware.

You can see it in the upper, left-hand corner there.

And that was when we were still at Stanford:

google.stanford.edu.

But when we became google.com, we realized that a really

important part of what we did would be managing hardware

costs and building pieces of software that could survive

hardware failure.

So we built our own hardware.

And you can't really see it in this picture, but those hard

drives in the middle there are just regular hard drives like

you'd buy from Fry's or CompUSA.

They're probably not as good as what's in

your kid's game machine.

They break a lot.

So we had to build software to allow machines to break, and

to allow the software to handle the failures in a

really clever way.

We made a transformational innovation to run on

extraordinarily cheap hardware and build software and

computer science to handle that.

That changed the terms of the game.

We've recently published a paper that showed that our

cost per MIP, as opposed to reliable hardware, we get

about a 10,000-fold cost per price increase.

That's a totally game-changing kind of number.

However, that wasn't enough.

We took that transformational innovation and we

got better at it.

If you look at the upper left-hand corner of this,

that's a data center that we built in about 2000.

We still have a lot of the same kind of weird computer

stuff, right?

They're all still these single hardware blades.

We still have cheap hard dives in there, et cetera, but we

look a lot more professional.

The racks make more sense.

It's got good cable management.

I'm a kind of ops guy as well.

I like good cable management.

The color of the cables are all right.

Proud of that data center, with one minor problem.

It turns out that, if you put a bunch of computers in one

spot, they get hot.

So we had to add a fan.

We're not quite perfect yet.

We're getting there.

And, if you look, by 2001, we're building data centers:

look, no fans.

You know, we've gradually gotten better.

That's the story.

A transformational innovation merely starts the clock on

when and how you do incremental

innovations on top.

That article that said innovation, for its own sake,

is useless, that article was right because a

transformational innovation is only as good as the business

value it creates, and only as useful as the incremental

innovations that happen after it to make it better and make

the economic value sustained.

At this point in the talk, I normally have either half my

audience asleep, which appears to be about right, or I have

people saying, "Yeah, that's great.

So what?

Tell me how to do this.

I don't understand.

I like the idea of innovation.

We'd love to have a bunch more.

Go buy me some of that."

Innovation, for its own sake, is useless.

Yes, you're right.

Innovation based on what users need is likely to create

economic value, so, whatever you do, start

innovating with the user.

And, obviously, the first thing you should do is ask

users what they want, right, Mr. Ford?

Yeah, right.

It won't work.

If I'd have asked customers what they wanted, they'd have

asked for a faster horse.

For those of you who don't know, Henry Ford built cars,

been dead a while.

Anyway, he didn't build faster horses.

He's right.

Users don't know what they want, ever.

But they might know what problems they have. So,

Googlers talk a lot.

You may have noticed that from this presentation.

Sorry.

We send lots of emails too, lots of them.

I get several hundred emails a day that aren't spam.

So that problem's been around for a long time.

And we had an engineer who was getting annoyed with it.

He was tired of the volume of email.

Stop there.

Let's take a step back.

Everyone knows that email is a solved problem, right?

We released Gmail on April 1st a couple of years ago, an idea

which sounded brilliant and funny at the time and turned

out to be most unfortunate.

It was not an April Fool's joke, we really meant it.

We just thought it was funny to release it on April 1st.

When we released Gmail, everyone email

was a solved problem.

All you want is folders, and a rich interface, and lots of

colors, and this idea that, maybe, you'll have a megabyte

or two, probably, actually, a couple of hundred kilobytes of

email at any one time.

Email was a solved problem.

Now, let's go back to this engineer who's getting annoyed

about all the hundreds of emails he's getting a day.

He comes up with a pretty simple idea, actually.

Wouldn't it be helpful if we just sorted all the email by

the thread that it's about?

So, if I get 50 emails about my funny-looking, purple

shirt, wouldn't it be nice to have all those 50 together so

I can delete them all at once?

Or 50 emails about how my hair is too long, or pick your

poison, put them all together, we call it conversation view.

And second, wouldn't it be cool if it were possible to

store all your email online and search it?

Neither of those two ideas was in the email solution I

described a minute ago, right?

And yet, they're the single most popular and important

parts of our Gmail product: conversation view, and,

essentially, infinite email storage and mail.

This is an example of the kind of innovation that happens

when your users try to solve their own problem.

I used Gmail as an example because it's a very compelling

example, but I could have used dozens of examples.

Fundamentally, your users don't know what they want, so

you can't ask them.

If you ask them, they'll give you a dumb answer.

But, if you can identify the problems they have, you can

find interesting products.

So that's the first way of listening to your user.

Listen to your user by starting with their problems.

So, for example, when we launched Google News,

everybody understood that the news on the web was going to

be the province of the online repositories of these

tremendously great content producers, and that the web

was going to be another outlet for really creative content.

And so the only way to do news interestingly on the web was

to do content creation.

We don't do content creation.

We don't do editing.

There's tremendously talented people who write articles, and

edit articles, and make really, really important

content get created.

And what we do is allow people to find it more easily and

aggregate them together.

Google News is now one of the most popular news sites on the

web with no content creators or editors.

We started with a problem, I can't find a new story, and we

created an incredibly important product.

The same is true of Desktop Search or Google Local.

Start with the users.

Start with problems. Don't ask your users because

they won't get it.

But it turns out your users are talking to you every day.

Your users are talking to you every day through what they

actually do.

Another way to listen to your users is listen to the data.

OK, this is the interactive part of the

talk, boys and girls.

Everybody wake up.

Oh, yes.

Move around, and very nice, woo.

Move with your fingers.

Wiggle.

Wiggle.

Wiggle your fingers.

OK, good.

What's on the screen?

No, it's not cities.

What's on the screen is, clearly, a map of the world.

It is not a map of the world at night.

It's obviously not a map of the world at night because the

entire world is not at night at one time.

So, barring the sun burning out or something really creepy

happening, it's not that.

We put a white dot on the map every place we

saw a query, OK?

So, what that means is that, someplace in there, there's

somebody doing a query.

Obviously, it makes sense.

You can, sort of, see the cities, because there are more

people in cities, thus, there are more queries, et cetera.

And you can, sort of, see a temporal effect, right?

Like, bright, bright, bright, so, clearly, people wake up in

Europe, and they wake up in the East coast, and they wake

up on the West coast, et cetera.

You can see, kind of, Japan over here.

What can't you see?

Where's Africa?

I promise it's there.

I flew over it a few days ago, the

continent was there, really.

Why are there no searches in Africa?

So, what we just learned from this, what we just learned

from listening to our users is that we don't

have users in Africa.

What causes our user problems in Africa?

Fundamentally, what causes the user issues in Africa is very,

very bad wireline internet.

Governments have learned two things about the

internet over the years.

Thing number one: profoundly destabilizing to authoritarian

governments.

Problem number two: really great source of tax revenue.

So, fundamentally, wireline access in Africa is tightly

controlled by the government and heavily taxed, thus, not

growing that fast.

So, for example, if your company's mission is to make

all the world's information universally accessible and

useful, you're going to have to try a different tack, a

different set of products, a different set of goals, to

deal with Africa.

And so, for example, recently, we announced a set of

partnerships with universities in Kenya to provide

satellite-connected internet.

And we're working with a series of mobile providers to

try to provide mobile access.

What we learned here is there's a huge need for

products from users that couldn't talk to us because

they can't reach us.

Let's continue looking at what our users tell us.

Our users tell us a lot every day.

Ignore the scale and the times, don't worry about it.

Fundamentally, what you have here are lines that show query

usage: how many queries are coming to

Google, more or less.

On the left-hand side is a normal Sunday.

And you notice that people are asleep.

Do, ta, do, ta, do, they wake up.

They do some queries.

They kind of hang around, then they go

play golf, or whatever.

Maybe this is the 60 Minute spike or something, when

there's something on the news and you're trying to

understand it, you know.

Anyway, the usage: low usage.

On one particular Sunday, we see this pattern.

What's different?

This little spike.

What do we think the spike is?

Yes, the spike is a costume malfunction at the Super Bowl.

That would be the Janet Jackson spike.

Continuing on.

Wandering through our query logs.

This one's kind of fun, if you're a news company.

December 22, in 2003: different scale.

Don't try to pay attention to the scale, right?

Everyone's asleep.

Everything's asleep.

Everything's asleep.

The width of this bar is one hour.

Nothing happening.

Nothing happening.

Nothing happening.

Oh, my god.

Something happening.

Pooh.

Nothing happening.

Nothing happening.

Nothing happening.

What happened?

There was an earthquake in central California that was

felt as far east as the Mississippi, and as far north

as Canada, and as far south as Mexico.

So, fundamentally, a lot of people felt the earthquake.

What happens when you feel an earthquake?

You want to go find out what it is.

So what do you do?

Everyone goes to Google, and does a query to try and find

that site that gives you earthquake data, which is the

USGS, by the way.

They clicked on the top link, which is the USGS, and

promptly denial of service attacked

the USGS out of existence.

But then they settle down.

And you see, over time, they figure out, oh, it's just an

earthquake.

No big deal.

Why is this interesting?

This is interesting because, fundamentally, the news sites

are in this spike too, all right?

The news sites don't have data yet.

They're still trying to figure out, they're scrambling

reporters, they're trying to understand what's going on.

Whereas, a lot of the people who those news sites want

reach, the people who care about what just happened,

already know.

So the democratization of information is, in some sense,

working against traditional news creation because a lot of

the people that you want to talk on the news know the

answer already.

So the people you're talking to are the people who weren't

that interested in the first place.

If you're a news content creator, that kind of graph

makes you think hard about what is my

value in this ecosystem?

It might drive you to a new kind of product mix: a more

editorial product mix, or a more integrative product mix.

Fact reporting is hard here.

Google does a thing called the Google Zeitgeist. Google

Zeitgeist measures search terms that are rapidly moving

up or down.

They're much more popular than you'd expect, or much less

popular than you'd expect: just, sort

of, interesting trends.

For some reason, all of you people are fascinated with

Britney Spears.

Why?

I mean, seriously.

Anyway, I guess that's not my place to ask.

It turns out, in addition to having relatively little

talent compared to the number of albums she's sold, Britney

Spears has a complex spelling of her name.

If you do a search for B- R- I- T- T- A- N- Y, Spears, that

is not her.

However, there's undoubtedly somebody on the web who

misspelled Britney Spears' name that way, so you'll

probably get a result or two.

And so we would notice in the query logs that someone did a

search for that, and then got a couple of results, and then

clicked back and did another search pretty quickly.

It's clearly not the right result.

They might then try, I don't know, B- R- I- T- N- Y, and

they get a couple of results.

But, you know, they'll wander around trying different

spellings until they stumble across B- R- I- T- N- E- Y,

the actual spelling of her name.

And now, all of a sudden, there's lots of results, OK?

So our algorithms noticed that people who were looking for B-

R- I- T- T- A- N- Y usually ended up doing a successful

search for B- R- I- T- N- E- Y.

What does that give us?

That gives us spelling correction.

We now know, we, being a bunch of computers, know, meaning

have learned through an automated and objective

fashion how to do spelling correction.

We can tell you, you probably misspelled that word, which,

basically, if you think about it, is automated search

refinement.

It's a little, tiny innovation, that was given to

us because we found a problem by listening to our users.

Again, nobody, no user, would have said, "Well, I need a

spelling correction," but, clearly, they do.

You can ask your users what they want

and get a dumb answer.

You can try and find problems your users have and make

interesting answers.

You can look at their data, and you can actually study

your users.

We do something interesting with our users.

We run eye-tracking studies, which is to say we hook people

up to a computer.

Not literally, they sit in front of the computer.

And we use this special apparatus, which follows their

eyes moving around, just like your eyes are probably moving

around following my hand.

And there's things called saccades, which have to do

with your eyes just moving.

But, fundamentally, if I watch what your eyes do, I can get a

sense for what you're thinking about.

So let me show you a little bit of an eye tracking video.

Now, the red dot that you see is where the person's eyes are

focused right now.

This person is doing a search.

They did a search for flowers.

They got a bunch of results.

They got some ads.

They're kind of focused on the ads up-top, and

they click on an ad.

Now, they're waiting.

And they got to a flower site.

And you can see they're kind of looking all

over heck and gone.

They can't figure out what to do.

Clearly, this site is confusing them, right?

Because they're looking everywhere trying to figure

out what to do.

OK, that was one example.

I'll give you another example.

I forgot what this example is.

Oh, I remember.

This person is going to be a spelling challenge.

They're actually looking for chocolate chip cookies.

You're going to have to trust me on this because they're

going to spend a long time trying to figure

out to spell chocolate.

Still thinking hard about it.

Still thinking hard about it.

And you can see, clearly, he's looking around.

This person's trying to figure it out, they're

struggling a lot.

OK, they've gotten part of the way to chocolate now.

They've gotten almost all the way.

And you can't see it, but they now have chip in there.

But what you can see is that there's a lot of visual

activity while they're doing a search.

What is this guy doing?

But he finally does get chocolate chip recipe

answered, and now he's trying to figure out what to do.

So you see him scanning each result, right?

OK, there's there.

There's that one.

There's that one.

There's that one.

Looking up and down.

This person is very, very thorough, so what we can learn

from this person is result ordering matters, et cetera.

And finally, they select a link and go on.

I'm not going to take you through this forever.

This particular site brought up a pop-up ad, which, if you

notice, the person completely ignored.

Now, I'm not going to take you through more of that, but this

is just another way to listen to your users.

So I've talked a lot about the structure of innovation, and

I've talked a lot about how you can get ideas for

innovation.

But one of the most interesting questions is how

do you build an environment that supports innovation?

That's what, I think, we got right.

We built a culture and a community of practice, from

the beginning, which focuses on ensuring that we keep this

innovation engine going.

Remember, I talked about transformational innovations

aren't enough.

Our auction wasn't enough, you have to do a whole bunch of

incremental innovations on top of that.

And, for any business of great value long-term, they have to

create additional transformational innovations.

So we have a whole bunch of elements of our culture which

are deeply embedded in ensuring we can create a

climate of information.

Most importantly, we live out loud.

We talk about everything.

We'll talk about performance.

We'll talk about technology.

We have arguments all the time.

Not really mean arguments, but arguments because we believe

that the best way to find a new idea is to get different

people thinking about the same problem.

We got our users to tell us what their problems were by

looking at the data, or studying them, et cetera.

And then we got a bunch of very smart people to have

different opinions and talk about them.

And so we try to hire people who have different backgrounds

and different ways of thinking, and put them in

rooms together.

And we pack them in pretty tight in most offices so that

you get great innovative energy and lots of different

perspectives.

It's the reason we serve food.

We don't serve food because we like having fat employees,

although, that's a perfectly fine side effect.

We also provide gym memberships.

Anyway, we provide food because what do you do around

the lunch table?

What do you do around a dinner table?

You talk, you engage, you interact.

And we'd usually serve it at these big, long

cafeteria-style tables so that you don't actually notice,

necessarily, who you're sitting next to.

We don't serve them in small tables where it's really easy

to have a discussion with just your clique, right?

It's a long table.

And so the person sitting next to you is going to weigh-in on

your input.

It's going to provide her thoughts on whatever it is you

happen to be talking about.

Lots of input on the same problem, lots of diverse

perspectives, living out loud drives innovation.

And I could have talked about any number of

different kinds of things.

But one other one that really matters is, if you're going to

be an innovative company, you have to reward risk and not

punish failure.

Most innovations fail.

Most evolutionary changes are irrelevant--

you never notice--

or don't make any difference.

My hair color's brown, not black, it makes no material

difference to my life.

Similarly, most innovations in business don't matter.

Most are not actively destructive, they're just

irrelevant.

Those are failures.

For cultures that punish you for failing, what will rapidly

happen to the people who have ideas?

They will rapidly stop sharing them because they're

incented not to.

The hardest part of cultural change is, actually, to accept

failure as part of the learning process and actually

build support structures and processes around

learning from failures.

We've done that a lot.

We experiment a lot, and our value is in failing fast. Let

me give you an example.

We have a product called Google Checkout.

Google Checkout is a way that people can enter their credit

card and their buying information in Google and use

it on sites all over the web.

It's been a really important product for us.

It turns out, roughly, two out of every three shopping carts

with something in them on the web are abandoned.

Let me say that in action-oriented sentences.

You go to a web site that sells something.

You found the site somehow.

Maybe you did a search for it.

So you did a search for Doublemint gum.

You found a site that sells gum.

You went to that site.

You added the pack of gum to your shopping cart.

You clicked checkout.

So, you say OK, I want to complete this transaction.

I want to give you money at gum.com.

And two out of every three of those

carts are then abandoned.

So, if you think about that from the merchant's

perspective, that's the worst of all possible worlds.

I've gone to all the trouble of finding you, and getting

you, and interacting with you.

And you've used up my computer resources, et cetera.

You didn't give me any money.

And it's a clue that the user experience isn't very good

because you dropped out for some reason.

Something bad happened.

We built Checkout because, based on a set of user

studies, user surveys, and user data, we found that there

were two barriers to those shopping cart creation.

One was that users didn't want to give their private

information just from arbitrary websites.

And two, it takes an average 17 fields to fill-in an order

at an arbitrary website.

So we [UNINTELLIGIBLE]

Checkout so you give your information to us once, and it

takes you one click to checkout.

So we found that, on average, a Google Checkout-enabled

merchant has 1/2 as many cart abandons as a

non-Checkout-enabled merchant.

So, clearly, we got something right.

But, as we were talking about this product, we had a lot of

problems about UI, user interface.

Exactly how do we show Checkout in our search

results, in our ads?

And we had a lot of ideas.

But, we had lots of ideas.

I knew the answer.

I absolutely was positive it had to do with a color and a

certain badge, I knew the answer.

I just wanted to go ahead and do it.

So, we did a study.

And compared to four or five different models, my model was

the worst. OK, I was wrong.

I hate being wrong.

But I was wrong.

We didn't use my model, we used the model that worked,

and that's OK.

It's OK to run experiments as long as you follow it with

data and do what the data says.

I'll give you another example.

Our users used to tell us they wanted 20 search results on

the main page instead of 10.

By default, you do a search, you get 10

results back, right?

10 results takes about 400 millisecond to respond.

So we heard all the people say yes, more results.

More results.

We said OK, fine.

So we gave them 20 results.

The top 10 results are the same, right?

Nothing material changes here.

We just added 10 more in the bottom.

Nothing interesting happens, right?

Instead of taking 400 milliseconds to respond, it

took us 600 milliseconds to respond.

And we lost 15% of our traffic.

15% of the searchers dropped because they were bored

waiting for the results.

200 milliseconds, they were bored.

It's OK.

We learned something: speed matters.

You also have to have a culture that lets people be

passionate about what they're doing.

So one of the things that we do is what's called 20% time.

Our engineers are expected to spend 20% of their time

working on something which isn't their main project, just

something they're interested in.

You can envisage this as being oh, every Friday I'm going to

work on that.

Engineers don't actually do that.

They, basically, work for a couple of months on a project,

and then work for a couple weeks on a 20% time project.

Or they bank it up for a year, and then do a month, or

something like that, right?

Most of the time, when I talk to other CIOs about this, what

I get as the result of oh, my god, that's

incredibly stupid, right?

It's such a huge tax on productivity.

How could you possibly do that?

The answer is twofold.

One, we get such a motivation boost out of it that the 80%

of time they're spending is way more productive than just

80% of a normal engineer.

And second, some of our very greatest products have come

out of 20% time.

Google News was a 20% time product.

Almost everything on Google Apps,

Gmail, was a 20% product.

The economic value created by the few of those 20% projects

more than pays for whatever the tax involved is.

And another really important process-oriented component

around innovation, lots of companies want to structure

their innovation.

They have an innovation czar.

And they have an innovation process person.

And, when you have an idea, you've get to submit a

business case, and it includes a budget.

And then, a committee of people will meet in a dark

room, smoking cigars, and decide whether your

innovation is worthy.

The problem is, by that point, the start-up energy is so

high, that you've taught most innovators to do something

else, particularly for incremental innovations that

are little, tiny changes.

But little, tiny changes matter, remember?

And, in the event you actually manage to survive with a

transformational innovation, you're going to kill the

incremental innovations on top of it by requiring all of this

start-up energy.

We have a different approach.

We let there be chaos.

20% time projects, lots of small projects, lots of

projects starting and stopping all the time, it

creates total chaos.

And then, periodically, we try to organize the chaos into

themes, focus areas, center strange attractors--

if you'll stick with the mathematics analogy--

and try very hard not to kill the fragile, little things.

We measure success over time, but we cull projects late.

Rather than culling projects at start-up, we have projects,

regardless, measure their percent growth:

week-over-week, month-over-month,

quarter-over-quarter, how many more users you have as a

percent of you had last time.

If you're losing users, your project's not very good.

If you're flat, you have an issue.

If you're going up, you're interesting.

And, as you go up enough, we start investing in you.

And you go from a little, tiny project into

a much bigger project.

We try to cull projects as late as we possibly can to

give innovation, which is a very, very fragile flower, as

long a Spring as possible to thrive.

And finally, you can get innovation by getting your

users to do it for you.

One of our core principles is that we have several thousand

wicked-smart developers.

There are several million in the world.

But our users do innovations for

themselves on our platform.

There are some constraints you have to do.

One of them is you have to unlock the user data.

You have to let users own their own data.

A classic business model is that, by having

user data, I own you.

You can never get out.

It's pure lock-in.

We believe something different.

We believe that by providing you value, that you can always

leave. Developers will trust us and users will trust us

because they'll know if our product starts to stink, they

can just walk away.

So by that, they're willing to take risks on us.

We can get other developers to develop on our platform.

And we found some really, really terrific examples, like

the Maps API.

Google Maps: a great product.

Do you know what makes it even better?

That we have a public API that people can use.

So if, for example, you're interested in trying to find

an apartment in San Diego, that application exists.

We didn't build the particular application,

which is on this slide.

We built the map state, everything else came, I think,

in this case, from Craigslist, which is a lightweight web.

And there's a little application written by

somebody who doesn't work for us, that overlays Craigslist

information on Google Maps so that you can

find things in context.

That's a cool innovation.

I didn't pay for it, but Maps traffic and Maps data got the

benefit of that innovation.

Similarly, personalization, I think I commented on

personalization being one of the really hard unsolved

problems. We have a product, called iGoogle, which allows

you to set up your Google homepage any way you want.

And you can build gadgets on top of it.

We built a bunch of gadgets.

There are something like 50,000 gadgets in the iGadget

directory, we didn't build very many of them, at all.

For example, this translates gadget in the lower, left-hand

corner, Where is Petco Park--

Petco Park is in San Diego, by the way--

you could auto-translate from English to French, written by

somebody named Rafael, who lives in Sydney.

I don't know why the Australians are doing English

to French translation, but this person did.

And it did a reasonable translation.

How cool is it to get innovation that someone else

does the work for and your platform benefits from?

And finally, and perhaps most importantly, you have to be

your own user.

Remember, I talked to you about Gmail as an example of a

product that got created because a user--

in this case, that Google programmer-- hated email?

At Google, we use our own products.

We use our own products because that means we're more

likely to see our user's problems more quickly and get

more useful help.

We recently migrated from a third-party calendar server to

Google Calendar.

It's a corporate version of the commercial program that

lots of people in the world use today.

We did it over a weekend, and we moved about 600,000 events

across that weekend.

Users in the internet file bugs with us.

They say things like, "Oh, Google Calendar didn't work to

create this event last week." That's not a very helpful bug

because I have no idea what happened.

In the 12 hours after I migrated Google, all of Google

to Google Calendar, we had over 1,000 bugs filed by

Google engineers.

And the bugs were things like, "The font is offset by one

pixel on this screen after this 17-keystroke event."

Engineers file good bugs.

By being our own user, we found bugs more quickly.

We were able to find those errors, and we created a whole

internal market around incremental innovation.

So, for example, now there's a tool which goes around and

scrapes the menus of all the cafes.

And you can specify I like this kind of meat, and it will

put it on your calendar which of the cafes are serving that

kind of meat.

It's stupid, right?

It has very little value, unless you

happen to be a foodie.

But it's a little, lightweight, trivial

innovation because we were our own users.

Now, where are we in the talk?

We've talked about what innovation is.

We've talked about how you can figure out what

innovation to do.

We've done a bunch of examples of the benefits.

At this point, people are asking the questions of OK,

well, what can I do?

I like Google's culture, it makes sense, but what can I do

to create a culture of innovation in my company?

Google had a big advantage, we were a Greenfield's operation.

We created our culture.

For places that didn't create our culture, there are,

basically, only two levers you have to push.

If you look at the literature on business success, and on

military success as well, you'll find that almost

everybody talks about leadership as the answer.

Leadership is the answer to everything.

When I was a researcher at RAND, I did a bunch of studies

into, among other things, very highly

effective military units.

And one of these I found is that leadership does not

appear to matter at all.

Leadership appears to be completely a red herring on

almost every front except on one.

Without strong leadership, you will not build diverse teams,

you will not hire diverse employees.

If you do not hire diverse employees, you will not get

multiple perspectives.

If you do not get multiple perspectives, your innovation

will die because you'll all see the problem the same way.

Leadership is critical to building diversity, and by

extension, innovation.

And then, second of all, your HR processes, probably, you're

compensating people and promoting them based on

execution and execution alone.

The problem with that is that means risk-takers get caught

the short end of the stick, right?

Because risk takers won't always execute, so they won't

get extra money, they won't get promoted.

They'll just sort of lag.

And soon, they will learn that to get promoted, and get more

money, and get more important, and get a corner office, and a

designated parking space, you must never try anything new.

If you're a culture trying to become an innovative culture,

these are your two levers.

Cheap speaker's trick: cheap speaker's trick being the so

what slide.

The so what slide always tells the audience the guy's about

to get off-stage, so everybody wake up because there might be

questions in a minute.

This is my so what slide, therefore, I must be ready to

get off-stage.

There are, fundamentally, three things in my talk that

are worth remembering.

Number one, innovation's happening in your organization

today, it's just, probably, you're killing it.

The odds are good.

Left to its own device, innovation is fragile.

It will not survive.

Your job, as a leader, is to find the innovations, foster

them, wait as long as you can to kill them because it's

likely you'll kill the good ones.

We tried to kill Gmail.

We tried to kill AdSense.

It's likely you'll kill the good ones because the really

good ones are antithetical to what you know how to do.

And then, finally, you have to accept risk.

You have to take a tax for failure and treat it as an

opportunity for learning.

This is where your leaders come in.

This is where your HR processes come in.

The Description of Innovation at Google