Daniel Levitin [ 27 SEP 2016 | Data Literacy| 1:13:33 ] I started writing the book in 2001, because my job was to teach students at McGill University how to think critically. Both undergraduates and graduate students. We used a little textbook, “How to Lie with Statistics” that was written in the 50s. And it’s a wonderful book, but all the examples came from the US Steel Annual Report from 1928, and things like that.
So I asked them to update the book for me, by going to the newspaper and the internet and TV and Radio Broadcasts and come to me with examples of ways in which either journalists or — forgive me — ad people, had distorted things.
I ended up with these huge boxes of clippings and thought, “I’ve got to write a book about this”. Because the other funny thing that’s happened between 2001 to now is there’s been kind of a transformation.
2001 is long ago enough that people were still getting most of their information from books and magazines and traditional newspapers.
Now — you probably read the recent survey — most people under the age of 25 get all their news from facebook. What’s happened in the interim is that nobody has stopped and told people, “you’ve got to evaluate whether the stuff you encounter is true or not.”
Nobody is doing it for you. The Internet doesn’t have any gatekeepers.
Humans evolved in a social environment. For tens of thousands of years we were living in groups of about 150 people, and the people you knew, your whole life, were that same group of people. They were your family and close friends, and we developed a non-confrontational, non-challenging way of being in order to live in these small groups.
One of the things that happens in large cities that doesn’t happen in small towns. . . in small towns you’re not going to have a plumber who’s going to cheat you. They can’t afford to. They need repeat business. But in a large city, you could go through every plumber in the city Yellow Pages or Internet Directory and never get the same one twice.
So they have no incentive to do a good job.
That leads to a complacency that everybody we encounter – we’ve got this non-confrontational view about how we want to interact with people (most of us) and so we hear a claim and we go, “oh well it must be right” and “I don’t want to upset the apple cart and challenge this person”.
That’s part of it.
The other part of it, a generation ago, we had gatekeepers. We had reporters and librarians and people who were keeping track of what was true and what wasn’t for us. But the information explosion has made it impossible for them to keep up.
Polls are complicated by the fact that they’re sensitive to the very question that the pollster asks. They’re sensitive to whether you’re getting an adequate sample — nobody would take a poll seriously if I wanted to know the average height of people and I stood in front of a basketball court, in a gymnasium at a high school where basketball players were playing. Nobody would take that poll seriously. That’s not a random sample.
How do you do a random sample of Americans today?
If you call people on the phone and you use landlines, it’s going to skew towards the older people. If you use cell phones only it will skew towards the younger. You can try stopping people on the street, but that’s skews towards people who are out on the street. Maybe not people who are at work.
You have to be very careful to get a random sample and then even then you’re damned because the kinds of people who will talk to a pollster, who will stop doing what they’re doing and actually respond, are a particular kind of person — not necessarily indicative of the country.
We’re a visual species, as well as a social species, and none of us humans were made to be able to look at a whole column or row of numbers and pick out a pattern. Which is why we make graphs and charts. The eyes are drawn to the differences that we see in these visual displays.
But the people who make the graphs are often times hoping that you won’t look too close or they don’t know better themselves. So they give you something that’s dramatically distorting the numbers that are actually there.
Sometimes intentionally, sometimes unintentionally.
All we have to do is take a minute to take a look, carefully, at some simple things. If you are looking at a graph, you’ve got a vertical axis and a horizontal axis and they’ve got tick marks, make sure that there are numbers there.
If they don’t put any numbers there, they can draw anything they want.
And if the numbers are there, they have to tell you what they are.
I have an example in the book of gross sales, for a company, and it doesn’t tell you whether it’s dollars or units sold. It doesn’t tell you what the time period is. You really know nothing. And the graph could be anything at all.
Suppose you’re Apple Computer and you’re doing a big presentation at the end of the quarter to investors and stock holders and you know that it’s going to be widely covered. . . and suppose for all previous quarters, quarter by quarter since it’s introduction, the iPhone sales were increasing.
Now, for the first time in history, the number of sales goes down. This would be a calamity. Right? You don’t want this to get out, but you’ve got to report the actual numbers, so what do you do?
You report the numbers, but instead of showing the graph of the actual sales per quarter, you report cumulative sales, the total number of sales of iPhones. As long as you’re selling even one unit, the line is going to still go up.
This is what Tim Cook did.
And most people didn’t notice, and he got away with it.
The basic principle here is, we have to do it by ourselves. Most of us feel intimidated by numbers, I’m not talking about anything more complicated than grade 6 arithmetic, it’s not even math. I’ll give you an example of plausibility. There was a widely reported story, by the anti-marijuana initiative folks, that since marijuana laws in California stopped being enforced 35 years ago, the number of marijuana smokers in the state doubled every year.
If you don’t stop and think about it, you just accept it. But you could work this out on a piece of paper, or even in your head. You make simple assumptions. So let’s assume that 35 years ago, the most conservative assumption possible is that there was only 1 marijuana smoker in all of California.
So year 1 there’s 1 and it’s doubling every year. So you’ve got
|. . .|
By the time you get up to 35 years, as it doubles it goes up very fast, you have 17 billion people. And the population of the world right now is 7 billion. Maybe 7.5. So: not plausible.
When you hear the claim 4 out of 5 dentists recommend Colgate, what does it make you think of with respect to, say, Crest? That maybe only one dentist is recommending it. Certainly it makes you think that dentists prefer Colgate to Crest.
Remember I told you that when you are looking at polls you need to figure out what question was being asked. What was being asked of the dentists was, “name all the toothpastes that you recommend”. So 4 out of 5 did recommend Colgate, but they also recommended Crest, and Gleem and Aquafresh and a whole bunch of others.
This is an example of someone distorting your frame on the picture. They’re deceiving you by the way they describe it.
This kind of stuff unfortunately happens all the time.
We’re so comfortable with averages that we don’t stop and think what they really are. They’re a distortion of the facts. You take a whole bunch of numbers, and you reduce them to one. So you’re going to have to be missing something about the picture when you do that.
As an example, suppose I tell you there’s a room over there right near us with ten people in it. The average net worth of people in that room is $5 billion dollars.
Depending on the kind of business you’re in, if you’re a banker or an investment advisor, or a seller of luxury cars, or maybe you’re an investigator with the IRS. That might be a room you’d want to go to. And spend some time with those people whose average net worth is $5 billion dollars. But what if I tell you it’s a Fortune 500 executive and 9 indigent people? The average is still $5 Billion. The executive is worth $50 Billion.
It doesn’t really tell you what’s really going on in the room, does it?
You have to be wary about an average. If you have a room full of NBA players and a room full of 3rd graders, the average height might be 5’2″, but that’s a number that nobody in the room actually has as their height.
You’ve got a bunch of 4 feet people and a bunch of 7 feet people.
Daniel J. Levitin is Dean of Social Sciences at the Minerva Schools at KGI in San Francisco and a faculty member at the Center for Executive Education in the Haas School of Business at UC Berkeley.