Practice English Speaking&Listening with: Losing Player Trust - The Data Dilemma - Extra Credits

So much of today's gaming is driven by hard data,

things that we developers can affix numbers to--

Sales,

play length, review scores...

but in focusing on that hard data

we have, at times, lost focus on all the things we can't track

by absolute numerical metrics:

Things like how much you trust or respect a company.

And, in the long run, that does matter.

So, here's something that happens way too often in this industry.

Let's say a game comes out--

Let's call it

Hecknomancer--

and it's great,

and it sells really well.

So, the company behind it will crank out a sequel.

Now, maybe Hecknomancer 2 isn't quite as good as the first one,

but because it's a sequel to a successful game

it will have a much bigger marketing budget this time.

And that, plus good residual word-of-mouth from the first game,

will lead to it selling more copies than the original.

Now, the review scores for this inferior sequel will be a little bit lower, of course.

Maybe not as low as they should be, because the hype is real and reviewers are human beings,

but lower all the same.

Hecknomancer 2's developers will see this dip in scores,

and somebody on the team will no doubt bring it up to the rest:

"Hey, those scores are looking worse than the last time."

but then, inevitably, this sort of internal criticism of the game will get shot down

when somebody points to the sales numbers,

because more people are buying this one.

They'll insist that the critics are just being elitist about it.

"I mean clearly we're doing something right, because people are into this.

Look at those sales numbers!"

And so, critique

gets dismissed as people flocked to the stores.

Then Hecknomancer 3: Revelations comes along,

with an even bigger marketing budget,

because now the thing is a proper franchise.

And because Hecknomancer 2 was the game of the season last year

even more people are gonna buy this one.

And once again those escalating sales numbers are pointed to as a defense against all internal critique,

and the company plows forward.

This is exactly how a great series gets driven into the ground.

Now obviously not all franchises are gonna

see that sort of linear decline in quality.

But in this scenario where review scores are slowly falling despite rising profits?

If you only look at the things that you have hard data for,

skimming the surface level of that data rather than

aggressively digging down into all of its root causes...

You won't see the real damage that diminishing quality is doing until it's too late.

Because that slow deterioration in quality actually *is* having an impact on sales.

It's just buried and hard to see underneath all that other data.

Fans do fall out of love with a series once it starts to wear thin, and given enough time,

the number of fans lost will simply outweigh what marketing money and name recognition can do.

This is where the idea of soft data comes in.

You'll often hear that you can't objectively measure the quality of a game.

And that's true, but we can measure something's relative quality.

We do that all the time.

Here - I'll show you.

Which is the better film: Mad Max: Fury Road, or The Emoji Movie?

Now, I may not get a hundred percent agreement on the answer to that question, but I will get pretty clear data.

It's relative data, it doesn't give me a number representing the amount of quality each film contains,

it just tells me how the two relate to each other. But that data is still valid.

The information it does convey is still helpful.

It still reflects the real world to the same degree that anything can,

and if we ask that question again with another data point, like...

Which is better, Spider-Man Homecoming or Mad Max: Fury Road?

Which is better, Spider-Man Homecoming or the Emoji Movie, and so on

then we can start building up an ordinal scale and that sort of relative, or positional scale,

can give us data that is incredibly helpful for judging the actual quality of our games.

Heck, just do that scale for games in the Call of Duty series, or all of the Assassin's Creed games

and I think you'll learn a lot.

While you're at it, throw Shadow of Mordor into that Assassin's Creed list as well,

because there's (whispers) probably some very useful information to be gleaned from that comparison.

But this goes beyond just rating how well liked games are. This extends to things like customer trust.

Trust is something that we basically never take measures of as an industry.

But, we probably should, it's relevant to a lot of scenarios.

Like - here.

Have you ever happened to see a company cram a bunch of particularly abusive micro transactions

into a game you might otherwise love?

Take your time. I'm sure you can think of an example.

Now, throwing those micro-transactions in almost certainly led to an increase in revenue.

Especially in the short term, and since all eyes are on that bottom line

including them will be seen as the right decision.

Even if players complain about it.

After all, the industry is all too aware of the fact -

- that players will very often continue to purchase and play the games we gripe about.

Which...

Fair, we do that.

But here's the problem:

Looking at just those hard metrics and the bottom line,

we basically have no way of actually telling what damage we might have just done.

I mean, maybe all of those complaints are just short-term griping from people who want everything for free!

And in this case, players are generally fine with the microtransactions.

It could be, or,

maybe putting those micro-transactions in the game actually *has* eroded your players' long-term trust.

And if so, that is a huge deal, even for the bottom line,

because people might not immediately quit, or even change their playing patterns over one offense like this...

But there is a point past which, if trust dips low enough,

players will just give up on a game or stop buying sequels,

and stop giving you money.

And in either case, don't you want to at least know?

You don't even have to get super sophisticated with it.

Imagine if, every three months,

you gave your players a survey asking them to rate how much they trusted your company.

Or, arguably even better -

imagine if we as an industry regularly asked players how much they trust a number of different game companies?

Just like that you would have a running timeline of your company's relative trust position.

It's not an absolute number,

but it does show your place relative to the rest of the industry

and gives you a sense of the long-term impact of some of your decisions.

So, if your company takes a sudden nosedive on that chart,

then you can review what you did last quarter and try to find the cause.

Or maybe some other company just shot up the chart all of a sudden...

What did they do?

Maybe you could take a closer look, crib some of their ideas.

When we talk about metrics and the use of metrics in this industry,

it is important that we look at all the hard data we've been gathering -

The things that are distinctly measurable with absolute non relative meaning.

But it's also important that we start finding better ways to gather some of that soft data,

the things that we can't affix an absolute value to,

but that still have very real meaning.

Because getting better at tracking that means happier customers,

better long-term health for our companies -

and hopefully, better games for everybody.

See you next week!