DOUGLAS MERRILL: Hi, I'm Douglas Merrill.
I want talk today about innovation at Google.
Thank you very much for coming.
I'm really excited to get the chance to talk about this.
We are an engine for innovation.
Yes, technically, we're a search, advertising, and
applications company, what we call search, ads, and apps.
But the reason we've been successful at search, ads, and
apps is because we've managed to innovate and find new and
unusual ways to solve really, really hard problems. And
that's what I want to talk about today.
Google was founded in the late '90s by two graduate students
with this amazing idea to take all of the world's information
and make it universally accessible and useful.
All of the world's information universally
accessible and useful.
The three parts of our mission that operate us today: all the
world's information, not just the web; all of the world's
information, not just the web in English; the web in any
language, books, print, radio, TV, all of the world's
Universally accessible, although I'm projecting this
deck today from a laptop, in lots of the world they don't
have good wireline internet access.
Or laptops are very expensive.
So, to make the information universally accessible, you
might need to go over a telephone, or you might need
to go over some other kind of kiosk device.
Universally accessible, all of the world should get to reach
out to this great new world.
And useful, the hardest part of making information is not
actually making it available, it's making it useful
once you have it.
You can present large tables of numbers that have all the
information value, but if you don't put them on a map, those
numbers are less useful, et cetera.
We spend a lot of our time, and a lot of our AI work, and
a lot of our innovation around figuring out ways to make more
Let me talk for a couple of seconds about what search is.
Google is a search company at heart.
We know how to provide search experience.
And there are really four parts of search.
And I want to talk about this, not to make you Google
experts, but rather because it helps provide context for some
of what I'm going to say later.
The first thing, when people talk about search--
you see this in press reports of search a lot--
they ask about how big is your index?
How many documents do you have?
And we don't talk about that a lot because it's not really
the right number.
We talk about comprehensiveness.
Comprehensiveness is more than just index size.
Comprehensiveness is how much of this information of a kind
do you cover?
So web comprehensiveness says how many web
pages do you have?
You could probably count them, but the counting's kind of
And so that's why we don't really engage in it.
Do you count the same page twice.
How do you decide?
The point is, if someone has a query that should be answered
by a web page, do you have that web page in your index?
The problem is, lots of queries shouldn't be answered
by a web page.
If you do a query, 'What is Winston Churchill's birthday,'
you ought to get a date back, not just a web page.
If you do a query that says, 'Where is Google
headquarters," you should get a map back, not
just a Google page.
Sometimes comprehensiveness means more
than just web pages.
And, if you think about those examples I just gave you,
relevance is also important.
Relevance is the second component we talk about in
Relevance is a term of art, meaning that the results that
appear close to the top are actually what you meant.
So, notice what I just said, "what you meant." Our goal is
to have the top result be not just what you asked for, but
what you should have asked for, what you really meant.
Let me give you an example.
If you do a query, you do a search for the word A- P- P-
L- E- S, the word apples, you're going to get a bunch of
results back, including stuff on fruits and all kinds
If you do a search for A- P- P- L- E, apple, you're going
to get another set of results back.
Mostly about a computer manufacturer because our
automated and objective algorithms have figured out
that users, when they do a search for Apple, are more
often looking for a computer manufacturer--
now a phone manufacturer, or an Mp3 player manufacturer--
than they are for a fruit.
When we talk about relevance, we mean the innovations that
innovations, note: foreshadowing a cheap
technique used by authors--
the innovations that we've done to make the results more
likely to be what you wanted not just what you said.
Another interesting thing, speed matters.
I'll talk about this a lot more later.
Latency matters a lot.
In fact, in a minute, I'm going to give you an actual
statistic that shows that we can lose 15% of our traffic
just by slowing down 200 milliseconds.
It turns out speed matters a lot.
Speed matters as much as anything else.
If you have a comprehensive, good, highly-relevant answer,
and it takes too long to respond,
your users will opt-out.
So we've done a whole bunch of technical work to make it
possible for us to answer any question, anywhere in the
world, in about 400 milliseconds.
Not a long time, unlike how long I'm
talking on this slide.
So I should pick up the pace.
The most important part of search is the user experience.
Do your users get what they want, how they understand it?
So recently, we announced One Google, or what we call
universal search, this idea that you can do a search for
Darth Vader and get, not just web pages about Darth Vader,
and maybe some images about Darth Vader, but also, maybe,
some video clips, and maybe some blog content,
and some news even.
Although, there hasn't been news about Darth Vader lately
that I'm aware of.
Maybe there has.
Is there a new Star Wars coming out?
Anyway, That's not relevant right now.
That was a pun to my previous slide here.
You guys are a tough crowd.
The most important part of the user experience is to make it
possible so that users understand, intuitively, what
That's what makes a good search engine.
Search is a very hard problem.
It's not enough to have a very big index.
An index of what?
You cover the right kinds of documents.
It's not enough to have a big index if you don't reply with
the right kinds of relevant, important, meaningful results.
It's not enough to have a big index and reply with the right
kind of meaningful, important, results if you do it slowly.
And finally, it's not enough to do all those things if you
do in a way that the user can't understand, and you
don't help the user have the experience of what they're
trying to find.
Those are the components of search.
And search is still not solved.
We've been working on it for a long time.
We have persons centuries of innovation into this to get
where we are today.
And yet, there are a lot of problems, that we're still
working on, that are really, really hard.
Google Book Search, for example, a really interesting
project we have going on that's partially with
publishers and partially with libraries to try and
understand the content that's in books.
We're scanning the content and making it so you can search.
And then, you can search for content which is in books.
There's lots of great content in books.
I like books.
Most people have read books.
Have you guys read a book?
They're both saying no.
Anyway, the point of a book search is to say there's
information which isn't on the web.
Remember our mission: all of the world's information
universally accessible and useful.
It's useful to be able to ask Google, "Who is Ishmael," and
get an answer back that says you should go buy Moby Dick
from Amazon or from your local book seller.
Access on mobile, I've talked a little bit about that.
I'll talk more about it later.
It turns out that there are millions of cell phones in
Africa with non-wireline internet access, and a handful
of wired internet access points.
If you want all of the world's information to be universally
accessible, you need to figure out the mobile access problem.
But, if you think about it, on a phone, you probably
shouldn't interact the same way you interact on a
You know, tri-touch is hard and the screens are very
small, so pages have to be re-rendered.
It's very complicated.
There ought to be something like, maybe, you call a phone
number and you speak into it, and you get
search results back.
Or maybe they text you the search results back so you can
choose between them.
There ought to be some way of recognizing as you move from
one medium to another.
So what that means is you went to Maps.
You found your start address.
You found your end address.
You did a search.
You printed it out, and then you walked off without it.
Why is it that that map just doesn't automatically follow
you to your phone?
It ought to be about you.
Problems like that, search is a fascinating thing across
Let's talk, briefly, about machine translation.
One of the very hardest AI problems, ever, is machine
translation: translating automatically from one
language to another language.
Google has recently won a series of awards from the
Association of Computing Machinery for automated
machine translation from English to Arabic and back,
and English to Chinese and back.
What that means is we have machines that analyze an
English stream, or a Chinese stream, and
translates it into the other.
I'm a very big soccer fan.
It turns out the rest of the world
calls that game football.
Americans call something else football.
When I do a search for soccer, it ought to be the case that
Google can return web pages, or news results, or video
results, in Italian about football, using whatever the
Italian word for football is, right?
Machine translation is another way to let people have access
to all of the world's information.
One of the greatest side effects of the internet
revolution has been the democratization of
information: the ability for people to tell their story
regardless of who they are.
In the past, history was always written by the winners.
Pop quiz: what's the difference between a
revolution and a civil war?
Answer: who won?
If the folks having the revolution won, it's a
If the government won, it's a civil war.
The only difference is who won.
History is written by the winners.
That's not the right answer.
History ought to get written by everybody.
And one of the great things about the internet, and
companies like Google, and others, is that we've given
tools for everyone to tell their story.
But it's not enough to be able to tell your story, you have
to be able to find that story.
Search is the oxygen of the information economy because,
without search, you can't find information.
If you can find information, you can find multiple
If you can find multiple perspectives, you get to have
different, more innovative, answers.
Google is an innovation engine, and we are applying
those innovations to these problems and others.
Somehow, you all have been nice enough to let me use the
word innovation, probably 1,000 times already, without
There are really three kinds of innovation.
Innovation's an interesting topic in the world.
Fundamentally, about every five or six years, there's a
wave. That wave is something like this, somebody publishes
a Forbes paper, or a Harvard Business Review article, or
something, that says, "Wow, companies need to innovate
more, because, if they innovate more,
they'll get new products.
They get new products, they'll get new markets, and they will
own the world.
A new hegemony will arise." And so there's a bunch of
articles about innovation and, often, references to hegemy--
Hegemonic kinds of companies.
I struggled a lot over that word.
I'll keep trying it.
I'll throw it in a few more times into the talk.
You guys can all laugh when I screw it up.
The problem is, that goes on for a while.
There's lots of focus, and lots of focus.
And then, suddenly, somebody writes the
The counterpoint article goes something like, "Oh, yeah.
Innovation is not useful.
Innovation just wastes money.
Fundamentally, a company should focus on their core
business because, clearly, innovation is antithetical to
And then, you're on the downward cycle.
And, all of a sudden, everyone's like, oh, we
We should execute better.
And there's this move back to scientific management.
And then, two or three years later, we
start it all over again.
So we're, sort of, over here, it seems to me,
on this wave, right?
We have had the recent articles in the popular
business press talking about innovation for its
own sake, et cetera.
The problem with the whole conversation is they also
don't define what innovation means.
The truth is, I agree.
Innovation, for its own sake, isn't useful.
So then, what is useful?
Let's step back.
What is innovation?
There are really, if you think about it, three kinds of
There's incremental innovation: a little thing
changes that changes something little, small.
There's a great biological example of this, right?
Sorry, for the creationists in the audience.
Anyway, evolution exists.
Evolution works on all this big thing of DNA, and makes a
And most of the changes are irrelevant.
And some of the changes are really bad.
And some of the changes are good.
But each one's little, tiny, incremental innovation.
It's also, in business, was that everything we talked
about in the '70s and early '80s around lean manufacturing
for cars, the idea that we should move the tools slightly
closer to the workers because that saved an extra movement,
and that saves a few minutes for a car; and quality is
free; and incremental innovations, small changes to
process; good to do.
But not the only kind of innovation.
There's also a second order of kind of innovation, what I
call, incremental innovation that has a side effect.
A biological example of a side effect: the opposable thumb.
If you think about it, animals farther down the evolutionary
ladder than us don't have opposable thumbs, they have an
extra finger up there.
And, over time, that finger moves over,
and over, and over.
And, all of a sudden, it gets far enough over to where its
axis of rotation is against the axis of rotation of the
other fingers, and you have an opposable thumb.
What happens the minute you have an opposable thumb?
You have tools.
You have pencils.
Suddenly, the world changes.
It wasn't that big a change, right?
It's just a few degrees of axis of rotation of a joint.
But that side effect creates civilization.
Incremental innovation with side
effects: very, very important.
And then there's the thing that we all think of as
innovation: transformational change.
Transformational change, suddenly something has
changed, and the world changes.
Everything is different.
Biological examples of
transformational change are hard.
You know, maybe it's the asteroid hitting in northern
Siberia causing the extinction of lots of species, maybe.
Or maybe it's the evolutionary change of your gills falling
off because the minute you don't have gills anymore you
better be able to move on dry land, or you're done, right?
Without gills in the water, you're drowning, yeah?
Yeah, you're awake, right?
Transformational things don't happen a lot in biology
because they're not very efficient,
they're not very effective.
They happen a fair amount in business.
I would argue that a transformational change in
business is Google's auction model for ads, where the price
for an ad is based on someone clicking on the ad.
And the value of that ad is the economic value created by
that ad clicking, which is actually the price bid by the
one under you.
So I would argue that the concept of targeted internet
advertising is a
transformational change to business.
But still, there aren't a lot of examples of it, and it's
what we all spend all of our time talking about.
Let me give you a few examples, in Google's context,
of all three kinds of these innovations, if I may.
You know, I'm a Google person talking about Google.
Thank you for bearing with me.
If you do a web search for the words "heart attack," I don't
really, actually, know what you're looking for.
If you were to grab a medical textbook, just an average
medical textbook, and open it up to heart attack in the
index, you're going to find a whole page of entries of
different kinds of things.
And each one refers to one or two pages.
So you're at an index entry for heart attack, and this
index has 100 sub-entries.
I don't know what you're trying to do.
So, originally, we noticed the same thing on the web.
People would do a search for heart attack.
And they go to a page.
And they come back up and they do a search for, oh, maybe,
heart attack symptoms, or heart attack treatment.
In fact, we have a story of a person who did a search for
heart attack, and looked at some pages, and then did a
search for heart attack symptoms, and then called 911
because he discovered that his symptoms matched, that he was
having a heart attack.
He went to the hospital and lived.
He was, in fact, having a heart attack.
It's a tremendously heartwarming story.
On the other hand, why make him do two
searches to find that?
We could have saved him, oh, I don't know,
30 seconds, or whatever.
So now, if you do a search for heart attack, what you'll see
is a little box.
And we called it-- down at the bottom---
what we call, search refinement.
These are other things that people who've searched for
heart attack have looked for.
We didn't have some human go through the logs
and figure it out.
We did an automated and objective analysis using
algorithms to study what happens in searches.
And we found that people who are looking for heart attack
are often looking for information for health
professionals, or causes and risk factors.
That's an incremental innovation, which adds a lot
of value to our users.
Incremental innovations are very useful.
We would have saved that guy an entire search.
However, we also have examples of incremental innovations
that include side effects.
So, here's a side effect.
Google has a business whereby we will syndicate ads, run our
ads on other people's content, it's called an AdSense.
In this case, you can see in the bottom, right-hand side of
the screen, we have some ads we're showing, and there's
also a banner ad on the top.
This particular page is about space exploration, so
presumably, someone did a search for space and got led
here, et cetera.
OK, now, that's sort of an interesting article.
And it talks a little about geology, so it's kind of a
good science article.
Our ads are prints of the space shuttle.
If you're interested in space, you might be interested in a
wall hanging and other kinds of things like that.
And also we have a newsy update on the Mars rover.
The random ad is an Army ad.
The connection between space travel and the Army escapes
The side effect here is that not a lot of people would
click on the Army ad.
That Army ad didn't add a lot of economic value to the
creator of this page.
However, lots of people click on our ads.
When someone clicks on a syndicated ad, if someone has
a content page and they show one of our ads, and a person
visits that page and clicks on an ad, we
get some of the revenue.
And the content creator gets some of the revenue, the
person who wrote that page.
What's the side effect?
The side effect is we made it possible for people to make
their entire living, or part of their living, writing
content online, sharing their stories.
I talk about the democratization of
information, everyone gets to tell their story.
Only rich people get to tell their stories full-time, rich
people or people who are getting paid.
We provided a way for people to get paid to tell their
stories in an effective way.
A side effect of this innovation was the creation of
a more democratized information field.
Really great side effect, so was the opposable thumb.
We also have transformational innovations that
have happened at Google.
Google started as a random assortment of hardware.
You can see it in the upper, left-hand corner there.
And that was when we were still at Stanford:
But when we became google.com, we realized that a really
important part of what we did would be managing hardware
costs and building pieces of software that could survive
So we built our own hardware.
And you can't really see it in this picture, but those hard
drives in the middle there are just regular hard drives like
you'd buy from Fry's or CompUSA.
They're probably not as good as what's in
your kid's game machine.
They break a lot.
So we had to build software to allow machines to break, and
to allow the software to handle the failures in a
really clever way.
We made a transformational innovation to run on
extraordinarily cheap hardware and build software and
computer science to handle that.
That changed the terms of the game.
We've recently published a paper that showed that our
cost per MIP, as opposed to reliable hardware, we get
about a 10,000-fold cost per price increase.
That's a totally game-changing kind of number.
However, that wasn't enough.
We took that transformational innovation and we
got better at it.
If you look at the upper left-hand corner of this,
that's a data center that we built in about 2000.
We still have a lot of the same kind of weird computer
They're all still these single hardware blades.
We still have cheap hard dives in there, et cetera, but we
look a lot more professional.
The racks make more sense.
It's got good cable management.
I'm a kind of ops guy as well.
I like good cable management.
The color of the cables are all right.
Proud of that data center, with one minor problem.
It turns out that, if you put a bunch of computers in one
spot, they get hot.
So we had to add a fan.
We're not quite perfect yet.
We're getting there.
And, if you look, by 2001, we're building data centers:
look, no fans.
You know, we've gradually gotten better.
That's the story.
A transformational innovation merely starts the clock on
when and how you do incremental
innovations on top.
That article that said innovation, for its own sake,
is useless, that article was right because a
transformational innovation is only as good as the business
value it creates, and only as useful as the incremental
innovations that happen after it to make it better and make
the economic value sustained.
At this point in the talk, I normally have either half my
audience asleep, which appears to be about right, or I have
people saying, "Yeah, that's great.
Tell me how to do this.
I don't understand.
I like the idea of innovation.
We'd love to have a bunch more.
Go buy me some of that."
Innovation, for its own sake, is useless.
Yes, you're right.
Innovation based on what users need is likely to create
economic value, so, whatever you do, start
innovating with the user.
And, obviously, the first thing you should do is ask
users what they want, right, Mr. Ford?
It won't work.
If I'd have asked customers what they wanted, they'd have
asked for a faster horse.
For those of you who don't know, Henry Ford built cars,
been dead a while.
Anyway, he didn't build faster horses.
Users don't know what they want, ever.
But they might know what problems they have. So,
Googlers talk a lot.
You may have noticed that from this presentation.
We send lots of emails too, lots of them.
I get several hundred emails a day that aren't spam.
So that problem's been around for a long time.
And we had an engineer who was getting annoyed with it.
He was tired of the volume of email.
Let's take a step back.
Everyone knows that email is a solved problem, right?
We released Gmail on April 1st a couple of years ago, an idea
which sounded brilliant and funny at the time and turned
out to be most unfortunate.
It was not an April Fool's joke, we really meant it.
We just thought it was funny to release it on April 1st.
When we released Gmail, everyone email
was a solved problem.
All you want is folders, and a rich interface, and lots of
colors, and this idea that, maybe, you'll have a megabyte
or two, probably, actually, a couple of hundred kilobytes of
email at any one time.
Email was a solved problem.
Now, let's go back to this engineer who's getting annoyed
about all the hundreds of emails he's getting a day.
He comes up with a pretty simple idea, actually.
Wouldn't it be helpful if we just sorted all the email by
the thread that it's about?
So, if I get 50 emails about my funny-looking, purple
shirt, wouldn't it be nice to have all those 50 together so
I can delete them all at once?
Or 50 emails about how my hair is too long, or pick your
poison, put them all together, we call it conversation view.
And second, wouldn't it be cool if it were possible to
store all your email online and search it?
Neither of those two ideas was in the email solution I
described a minute ago, right?
And yet, they're the single most popular and important
parts of our Gmail product: conversation view, and,
essentially, infinite email storage and mail.
This is an example of the kind of innovation that happens
when your users try to solve their own problem.
I used Gmail as an example because it's a very compelling
example, but I could have used dozens of examples.
Fundamentally, your users don't know what they want, so
you can't ask them.
If you ask them, they'll give you a dumb answer.
But, if you can identify the problems they have, you can
find interesting products.
So that's the first way of listening to your user.
Listen to your user by starting with their problems.
So, for example, when we launched Google News,
everybody understood that the news on the web was going to
be the province of the online repositories of these
tremendously great content producers, and that the web
was going to be another outlet for really creative content.
And so the only way to do news interestingly on the web was
to do content creation.
We don't do content creation.
We don't do editing.
There's tremendously talented people who write articles, and
edit articles, and make really, really important
content get created.
And what we do is allow people to find it more easily and
aggregate them together.
Google News is now one of the most popular news sites on the
web with no content creators or editors.
We started with a problem, I can't find a new story, and we
created an incredibly important product.
The same is true of Desktop Search or Google Local.
Start with the users.
Start with problems. Don't ask your users because
they won't get it.
But it turns out your users are talking to you every day.
Your users are talking to you every day through what they
Another way to listen to your users is listen to the data.
OK, this is the interactive part of the
talk, boys and girls.
Everybody wake up.
Move around, and very nice, woo.
Move with your fingers.
Wiggle your fingers.
What's on the screen?
No, it's not cities.
What's on the screen is, clearly, a map of the world.
It is not a map of the world at night.
It's obviously not a map of the world at night because the
entire world is not at night at one time.
So, barring the sun burning out or something really creepy
happening, it's not that.
We put a white dot on the map every place we
saw a query, OK?
So, what that means is that, someplace in there, there's
somebody doing a query.
Obviously, it makes sense.
You can, sort of, see the cities, because there are more
people in cities, thus, there are more queries, et cetera.
And you can, sort of, see a temporal effect, right?
Like, bright, bright, bright, so, clearly, people wake up in
Europe, and they wake up in the East coast, and they wake
up on the West coast, et cetera.
You can see, kind of, Japan over here.
What can't you see?
I promise it's there.
I flew over it a few days ago, the
continent was there, really.
Why are there no searches in Africa?
So, what we just learned from this, what we just learned
from listening to our users is that we don't
have users in Africa.
What causes our user problems in Africa?
Fundamentally, what causes the user issues in Africa is very,
very bad wireline internet.
Governments have learned two things about the
internet over the years.
Thing number one: profoundly destabilizing to authoritarian
Problem number two: really great source of tax revenue.
So, fundamentally, wireline access in Africa is tightly
controlled by the government and heavily taxed, thus, not
growing that fast.
So, for example, if your company's mission is to make
all the world's information universally accessible and
useful, you're going to have to try a different tack, a
different set of products, a different set of goals, to
deal with Africa.
And so, for example, recently, we announced a set of
partnerships with universities in Kenya to provide
And we're working with a series of mobile providers to
try to provide mobile access.
What we learned here is there's a huge need for
products from users that couldn't talk to us because
they can't reach us.
Let's continue looking at what our users tell us.
Our users tell us a lot every day.
Ignore the scale and the times, don't worry about it.
Fundamentally, what you have here are lines that show query
usage: how many queries are coming to
Google, more or less.
On the left-hand side is a normal Sunday.
And you notice that people are asleep.
Do, ta, do, ta, do, they wake up.
They do some queries.
They kind of hang around, then they go
play golf, or whatever.
Maybe this is the 60 Minute spike or something, when
there's something on the news and you're trying to
understand it, you know.
Anyway, the usage: low usage.
On one particular Sunday, we see this pattern.
This little spike.
What do we think the spike is?
Yes, the spike is a costume malfunction at the Super Bowl.
That would be the Janet Jackson spike.
Wandering through our query logs.
This one's kind of fun, if you're a news company.
December 22, in 2003: different scale.
Don't try to pay attention to the scale, right?
The width of this bar is one hour.
Oh, my god.
There was an earthquake in central California that was
felt as far east as the Mississippi, and as far north
as Canada, and as far south as Mexico.
So, fundamentally, a lot of people felt the earthquake.
What happens when you feel an earthquake?
You want to go find out what it is.
So what do you do?
Everyone goes to Google, and does a query to try and find
that site that gives you earthquake data, which is the
USGS, by the way.
They clicked on the top link, which is the USGS, and
promptly denial of service attacked
the USGS out of existence.
But then they settle down.
And you see, over time, they figure out, oh, it's just an
No big deal.
Why is this interesting?
This is interesting because, fundamentally, the news sites
are in this spike too, all right?
The news sites don't have data yet.
They're still trying to figure out, they're scrambling
reporters, they're trying to understand what's going on.
Whereas, a lot of the people who those news sites want
reach, the people who care about what just happened,
So the democratization of information is, in some sense,
working against traditional news creation because a lot of
the people that you want to talk on the news know the
So the people you're talking to are the people who weren't
that interested in the first place.
If you're a news content creator, that kind of graph
makes you think hard about what is my
value in this ecosystem?
It might drive you to a new kind of product mix: a more
editorial product mix, or a more integrative product mix.
Fact reporting is hard here.
Google does a thing called the Google Zeitgeist. Google
Zeitgeist measures search terms that are rapidly moving
up or down.
They're much more popular than you'd expect, or much less
popular than you'd expect: just, sort
of, interesting trends.
For some reason, all of you people are fascinated with
I mean, seriously.
Anyway, I guess that's not my place to ask.
It turns out, in addition to having relatively little
talent compared to the number of albums she's sold, Britney
Spears has a complex spelling of her name.
If you do a search for B- R- I- T- T- A- N- Y, Spears, that
is not her.
However, there's undoubtedly somebody on the web who
misspelled Britney Spears' name that way, so you'll
probably get a result or two.
And so we would notice in the query logs that someone did a
search for that, and then got a couple of results, and then
clicked back and did another search pretty quickly.
It's clearly not the right result.
They might then try, I don't know, B- R- I- T- N- Y, and
they get a couple of results.
But, you know, they'll wander around trying different
spellings until they stumble across B- R- I- T- N- E- Y,
the actual spelling of her name.
And now, all of a sudden, there's lots of results, OK?
So our algorithms noticed that people who were looking for B-
R- I- T- T- A- N- Y usually ended up doing a successful
search for B- R- I- T- N- E- Y.
What does that give us?
That gives us spelling correction.
We now know, we, being a bunch of computers, know, meaning
have learned through an automated and objective
fashion how to do spelling correction.
We can tell you, you probably misspelled that word, which,
basically, if you think about it, is automated search
It's a little, tiny innovation, that was given to
us because we found a problem by listening to our users.
Again, nobody, no user, would have said, "Well, I need a
spelling correction," but, clearly, they do.
You can ask your users what they want
and get a dumb answer.
You can try and find problems your users have and make
You can look at their data, and you can actually study
We do something interesting with our users.
We run eye-tracking studies, which is to say we hook people
up to a computer.
Not literally, they sit in front of the computer.
And we use this special apparatus, which follows their
eyes moving around, just like your eyes are probably moving
around following my hand.
And there's things called saccades, which have to do
with your eyes just moving.
But, fundamentally, if I watch what your eyes do, I can get a
sense for what you're thinking about.
So let me show you a little bit of an eye tracking video.
Now, the red dot that you see is where the person's eyes are
focused right now.
This person is doing a search.
They did a search for flowers.
They got a bunch of results.
They got some ads.
They're kind of focused on the ads up-top, and
they click on an ad.
Now, they're waiting.
And they got to a flower site.
And you can see they're kind of looking all
over heck and gone.
They can't figure out what to do.
Clearly, this site is confusing them, right?
Because they're looking everywhere trying to figure
out what to do.
OK, that was one example.
I'll give you another example.
I forgot what this example is.
Oh, I remember.
This person is going to be a spelling challenge.
They're actually looking for chocolate chip cookies.
You're going to have to trust me on this because they're
going to spend a long time trying to figure
out to spell chocolate.
Still thinking hard about it.
Still thinking hard about it.
And you can see, clearly, he's looking around.
This person's trying to figure it out, they're
struggling a lot.
OK, they've gotten part of the way to chocolate now.
They've gotten almost all the way.
And you can't see it, but they now have chip in there.
But what you can see is that there's a lot of visual
activity while they're doing a search.
What is this guy doing?
But he finally does get chocolate chip recipe
answered, and now he's trying to figure out what to do.
So you see him scanning each result, right?
OK, there's there.
There's that one.
There's that one.
There's that one.
Looking up and down.
This person is very, very thorough, so what we can learn
from this person is result ordering matters, et cetera.
And finally, they select a link and go on.
I'm not going to take you through this forever.
This particular site brought up a pop-up ad, which, if you
notice, the person completely ignored.
Now, I'm not going to take you through more of that, but this
is just another way to listen to your users.
So I've talked a lot about the structure of innovation, and
I've talked a lot about how you can get ideas for
But one of the most interesting questions is how
do you build an environment that supports innovation?
That's what, I think, we got right.
We built a culture and a community of practice, from
the beginning, which focuses on ensuring that we keep this
innovation engine going.
Remember, I talked about transformational innovations
Our auction wasn't enough, you have to do a whole bunch of
incremental innovations on top of that.
And, for any business of great value long-term, they have to
create additional transformational innovations.
So we have a whole bunch of elements of our culture which
are deeply embedded in ensuring we can create a
climate of information.
Most importantly, we live out loud.
We talk about everything.
We'll talk about performance.
We'll talk about technology.
We have arguments all the time.
Not really mean arguments, but arguments because we believe
that the best way to find a new idea is to get different
people thinking about the same problem.
We got our users to tell us what their problems were by
looking at the data, or studying them, et cetera.
And then we got a bunch of very smart people to have
different opinions and talk about them.
And so we try to hire people who have different backgrounds
and different ways of thinking, and put them in
And we pack them in pretty tight in most offices so that
you get great innovative energy and lots of different
It's the reason we serve food.
We don't serve food because we like having fat employees,
although, that's a perfectly fine side effect.
We also provide gym memberships.
Anyway, we provide food because what do you do around
the lunch table?
What do you do around a dinner table?
You talk, you engage, you interact.
And we'd usually serve it at these big, long
cafeteria-style tables so that you don't actually notice,
necessarily, who you're sitting next to.
We don't serve them in small tables where it's really easy
to have a discussion with just your clique, right?
It's a long table.
And so the person sitting next to you is going to weigh-in on
It's going to provide her thoughts on whatever it is you
happen to be talking about.
Lots of input on the same problem, lots of diverse
perspectives, living out loud drives innovation.
And I could have talked about any number of
different kinds of things.
But one other one that really matters is, if you're going to
be an innovative company, you have to reward risk and not
Most innovations fail.
Most evolutionary changes are irrelevant--
you never notice--
or don't make any difference.
My hair color's brown, not black, it makes no material
difference to my life.
Similarly, most innovations in business don't matter.
Most are not actively destructive, they're just
Those are failures.
For cultures that punish you for failing, what will rapidly
happen to the people who have ideas?
They will rapidly stop sharing them because they're
incented not to.
The hardest part of cultural change is, actually, to accept
failure as part of the learning process and actually
build support structures and processes around
learning from failures.
We've done that a lot.
We experiment a lot, and our value is in failing fast. Let
me give you an example.
We have a product called Google Checkout.
Google Checkout is a way that people can enter their credit
card and their buying information in Google and use
it on sites all over the web.
It's been a really important product for us.
It turns out, roughly, two out of every three shopping carts
with something in them on the web are abandoned.
Let me say that in action-oriented sentences.
You go to a web site that sells something.
You found the site somehow.
Maybe you did a search for it.
So you did a search for Doublemint gum.
You found a site that sells gum.
You went to that site.
You added the pack of gum to your shopping cart.
You clicked checkout.
So, you say OK, I want to complete this transaction.
I want to give you money at gum.com.
And two out of every three of those
carts are then abandoned.
So, if you think about that from the merchant's
perspective, that's the worst of all possible worlds.
I've gone to all the trouble of finding you, and getting
you, and interacting with you.
And you've used up my computer resources, et cetera.
You didn't give me any money.
And it's a clue that the user experience isn't very good
because you dropped out for some reason.
Something bad happened.
We built Checkout because, based on a set of user
studies, user surveys, and user data, we found that there
were two barriers to those shopping cart creation.
One was that users didn't want to give their private
information just from arbitrary websites.
And two, it takes an average 17 fields to fill-in an order
at an arbitrary website.
So we [UNINTELLIGIBLE]
Checkout so you give your information to us once, and it
takes you one click to checkout.
So we found that, on average, a Google Checkout-enabled
merchant has 1/2 as many cart abandons as a
So, clearly, we got something right.
But, as we were talking about this product, we had a lot of
problems about UI, user interface.
Exactly how do we show Checkout in our search
results, in our ads?
And we had a lot of ideas.
But, we had lots of ideas.
I knew the answer.
I absolutely was positive it had to do with a color and a
certain badge, I knew the answer.
I just wanted to go ahead and do it.
So, we did a study.
And compared to four or five different models, my model was
the worst. OK, I was wrong.
I hate being wrong.
But I was wrong.
We didn't use my model, we used the model that worked,
and that's OK.
It's OK to run experiments as long as you follow it with
data and do what the data says.
I'll give you another example.
Our users used to tell us they wanted 20 search results on
the main page instead of 10.
By default, you do a search, you get 10
results back, right?
10 results takes about 400 millisecond to respond.
So we heard all the people say yes, more results.
We said OK, fine.
So we gave them 20 results.
The top 10 results are the same, right?
Nothing material changes here.
We just added 10 more in the bottom.
Nothing interesting happens, right?
Instead of taking 400 milliseconds to respond, it
took us 600 milliseconds to respond.
And we lost 15% of our traffic.
15% of the searchers dropped because they were bored
waiting for the results.
200 milliseconds, they were bored.
We learned something: speed matters.
You also have to have a culture that lets people be
passionate about what they're doing.
So one of the things that we do is what's called 20% time.
Our engineers are expected to spend 20% of their time
working on something which isn't their main project, just
something they're interested in.
You can envisage this as being oh, every Friday I'm going to
work on that.
Engineers don't actually do that.
They, basically, work for a couple of months on a project,
and then work for a couple weeks on a 20% time project.
Or they bank it up for a year, and then do a month, or
something like that, right?
Most of the time, when I talk to other CIOs about this, what
I get as the result of oh, my god, that's
incredibly stupid, right?
It's such a huge tax on productivity.
How could you possibly do that?
The answer is twofold.
One, we get such a motivation boost out of it that the 80%
of time they're spending is way more productive than just
80% of a normal engineer.
And second, some of our very greatest products have come
out of 20% time.
Google News was a 20% time product.
Almost everything on Google Apps,
Gmail, was a 20% product.
The economic value created by the few of those 20% projects
more than pays for whatever the tax involved is.
And another really important process-oriented component
around innovation, lots of companies want to structure
They have an innovation czar.
And they have an innovation process person.
And, when you have an idea, you've get to submit a
business case, and it includes a budget.
And then, a committee of people will meet in a dark
room, smoking cigars, and decide whether your
innovation is worthy.
The problem is, by that point, the start-up energy is so
high, that you've taught most innovators to do something
else, particularly for incremental innovations that
are little, tiny changes.
But little, tiny changes matter, remember?
And, in the event you actually manage to survive with a
transformational innovation, you're going to kill the
incremental innovations on top of it by requiring all of this
We have a different approach.
We let there be chaos.
20% time projects, lots of small projects, lots of
projects starting and stopping all the time, it
creates total chaos.
And then, periodically, we try to organize the chaos into
themes, focus areas, center strange attractors--
if you'll stick with the mathematics analogy--
and try very hard not to kill the fragile, little things.
We measure success over time, but we cull projects late.
Rather than culling projects at start-up, we have projects,
regardless, measure their percent growth:
quarter-over-quarter, how many more users you have as a
percent of you had last time.
If you're losing users, your project's not very good.
If you're flat, you have an issue.
If you're going up, you're interesting.
And, as you go up enough, we start investing in you.
And you go from a little, tiny project into
a much bigger project.
We try to cull projects as late as we possibly can to
give innovation, which is a very, very fragile flower, as
long a Spring as possible to thrive.
And finally, you can get innovation by getting your
users to do it for you.
One of our core principles is that we have several thousand
There are several million in the world.
But our users do innovations for
themselves on our platform.
There are some constraints you have to do.
One of them is you have to unlock the user data.
You have to let users own their own data.
A classic business model is that, by having
user data, I own you.
You can never get out.
It's pure lock-in.
We believe something different.
We believe that by providing you value, that you can always
leave. Developers will trust us and users will trust us
because they'll know if our product starts to stink, they
can just walk away.
So by that, they're willing to take risks on us.
We can get other developers to develop on our platform.
And we found some really, really terrific examples, like
the Maps API.
Google Maps: a great product.
Do you know what makes it even better?
That we have a public API that people can use.
So if, for example, you're interested in trying to find
an apartment in San Diego, that application exists.
We didn't build the particular application,
which is on this slide.
We built the map state, everything else came, I think,
in this case, from Craigslist, which is a lightweight web.
And there's a little application written by
somebody who doesn't work for us, that overlays Craigslist
information on Google Maps so that you can
find things in context.
That's a cool innovation.
I didn't pay for it, but Maps traffic and Maps data got the
benefit of that innovation.
Similarly, personalization, I think I commented on
personalization being one of the really hard unsolved
problems. We have a product, called iGoogle, which allows
you to set up your Google homepage any way you want.
And you can build gadgets on top of it.
We built a bunch of gadgets.
There are something like 50,000 gadgets in the iGadget
directory, we didn't build very many of them, at all.
For example, this translates gadget in the lower, left-hand
corner, Where is Petco Park--
Petco Park is in San Diego, by the way--
you could auto-translate from English to French, written by
somebody named Rafael, who lives in Sydney.
I don't know why the Australians are doing English
to French translation, but this person did.
And it did a reasonable translation.
How cool is it to get innovation that someone else
does the work for and your platform benefits from?
And finally, and perhaps most importantly, you have to be
your own user.
Remember, I talked to you about Gmail as an example of a
product that got created because a user--
in this case, that Google programmer-- hated email?
At Google, we use our own products.
We use our own products because that means we're more
likely to see our user's problems more quickly and get
more useful help.
We recently migrated from a third-party calendar server to
It's a corporate version of the commercial program that
lots of people in the world use today.
We did it over a weekend, and we moved about 600,000 events
across that weekend.
Users in the internet file bugs with us.
They say things like, "Oh, Google Calendar didn't work to
create this event last week." That's not a very helpful bug
because I have no idea what happened.
In the 12 hours after I migrated Google, all of Google
to Google Calendar, we had over 1,000 bugs filed by
And the bugs were things like, "The font is offset by one
pixel on this screen after this 17-keystroke event."
Engineers file good bugs.
By being our own user, we found bugs more quickly.
We were able to find those errors, and we created a whole
internal market around incremental innovation.
So, for example, now there's a tool which goes around and
scrapes the menus of all the cafes.
And you can specify I like this kind of meat, and it will
put it on your calendar which of the cafes are serving that
kind of meat.
It's stupid, right?
It has very little value, unless you
happen to be a foodie.
But it's a little, lightweight, trivial
innovation because we were our own users.
Now, where are we in the talk?
We've talked about what innovation is.
We've talked about how you can figure out what
innovation to do.
We've done a bunch of examples of the benefits.
At this point, people are asking the questions of OK,
well, what can I do?
I like Google's culture, it makes sense, but what can I do
to create a culture of innovation in my company?
Google had a big advantage, we were a Greenfield's operation.
We created our culture.
For places that didn't create our culture, there are,
basically, only two levers you have to push.
If you look at the literature on business success, and on
military success as well, you'll find that almost
everybody talks about leadership as the answer.
Leadership is the answer to everything.
When I was a researcher at RAND, I did a bunch of studies
into, among other things, very highly
effective military units.
And one of these I found is that leadership does not
appear to matter at all.
Leadership appears to be completely a red herring on
almost every front except on one.
Without strong leadership, you will not build diverse teams,
you will not hire diverse employees.
If you do not hire diverse employees, you will not get
If you do not get multiple perspectives, your innovation
will die because you'll all see the problem the same way.
Leadership is critical to building diversity, and by
And then, second of all, your HR processes, probably, you're
compensating people and promoting them based on
execution and execution alone.
The problem with that is that means risk-takers get caught
the short end of the stick, right?
Because risk takers won't always execute, so they won't
get extra money, they won't get promoted.
They'll just sort of lag.
And soon, they will learn that to get promoted, and get more
money, and get more important, and get a corner office, and a
designated parking space, you must never try anything new.
If you're a culture trying to become an innovative culture,
these are your two levers.
Cheap speaker's trick: cheap speaker's trick being the so
The so what slide always tells the audience the guy's about
to get off-stage, so everybody wake up because there might be
questions in a minute.
This is my so what slide, therefore, I must be ready to
There are, fundamentally, three things in my talk that
are worth remembering.
Number one, innovation's happening in your organization
today, it's just, probably, you're killing it.
The odds are good.
Left to its own device, innovation is fragile.
It will not survive.
Your job, as a leader, is to find the innovations, foster
them, wait as long as you can to kill them because it's
likely you'll kill the good ones.
We tried to kill Gmail.
We tried to kill AdSense.
It's likely you'll kill the good ones because the really
good ones are antithetical to what you know how to do.
And then, finally, you have to accept risk.
You have to take a tax for failure and treat it as an
opportunity for learning.
This is where your leaders come in.
This is where your HR processes come in.