We can test this prediction to see if it is true. Consider running foot races, especially the marathon, one of the oldest and most popular sports events in history. The race covers 26 miles and 385 yards. It was introduced as an original Olympic event in 1896, roughly fifteen hundred years after—legend has it—Pheidippides ran to his home in Athens from the battlefield of Marathon, where his countrymen had just defeated the Persians. When Pheidippides arrived, he proclaimed, “We have won!” He then dropped dead.
FIGURE 3-5
The paradox of skill leads to clustered results
Source: Analysis by author.
John Brenkus, host of Sports Science for the television network ESPN, speculates on the limits of human performance in his book The Perfection Point. After giving consideration to a multitude of physical factors, he concludes that the fastest time that a human can ever run a marathon is 1 hour, 57 minutes, and 58 seconds.11 As I write this, the world record, held by Patrick Makau of Kenya, is 2 hours, 3 minutes, and 38 seconds. So Makau's record is 5 minutes and 40 seconds slower than what is theoretically possible, according to Brenkus.
Figure 3-6 shows two results from each men's Olympic marathon from 1932 to 2008. The first is the time of the winner. That time dropped by about twenty-five minutes during those years. This translates into a pace that is almost one minute faster each mile, which (as you runners out there know) is a substantial increase, even considering that it was achieved over three-quarters of a century. The figure also shows the difference between the time of the gold medalist and the man who came in twentieth. As the paradox of skill predicts, that time has narrowed from close to forty minutes in 1932 to around nine minutes in 2008. So as everyone's skill has improved, the performance of the person who finished in twentieth place and the winner has converged.
FIGURE 3-6
Men's Olympic marathon times and the paradox of skill
Source: www.olympicgamesmarathon.com and analysis by author.
The two-jar model shows that luck can overwhelm skill in the short term if the variance of the distribution of luck is larger than the variance of the distribution of skill. In other words, if everyone gets better at something, luck plays a more important role in determining who wins. Let's return to that model now.
The Ingredients of an Outlier
Note that the extreme values in the two-jar model are −7 and 7. The only way to get those values is to combine the worst skill with the worst luck or the best skill with the best luck. Since the poorest performers generally die off in a competitive environment, we'll concentrate on the best. The basic argument is easy to summarize: great success combines skill with a lot of luck. You can't get there by relying on either skill or luck alone. You need both.
This is one of the central themes in Malcolm Gladwell's book, Outliers. As one of his examples, Gladwell tells the story of Bill Joy, the billionaire cofounder of Sun Microsystems, who is now a partner in the venture capital firm Kleiner Perkins. Joy was always exceptionally bright. He scored a perfect 800 on the math section of the SAT and entered the University of Michigan at the age of sixteen. To his good fortune, Michigan had one of the few computers in the country that had a keyboard and screen. Everywhere else, people who wanted to use a computer had to feed punched cards into the machine to get it to do anything (or more likely, wait for a technician to do it). Joy spent an enormous amount of time learning to write programs in college, giving him an edge when he entered the PhD program for computer science at the University of California, Berkeley. By the time he had completed his studies at Berkeley, he had about ten thousand hours of practice in writing computer code.12 But it was the combination of his skill and good luck that allowed him to start a software company and accrue his substantial net worth. He could have been just as smart and gone to a college that had no interactive computers. To succeed, Joy needed to draw winning numbers from both jars.
Gladwell argues that the lore of success too often dwells on an individual's personal qualities, focusing on how grit and talent paved the way to the top. But a closer examination always reveals the substantial role that luck played. If history is written by the winners, history is also written about the winners, because we like to see clear cause and effect. Luck is boring as the driving force in a story. So when talking about success, we tend to place too much emphasis on skill and not enough on luck. Luck is there, though, if you look. A full account of these stories of success shows, as Gladwell puts it, that “outliers reached their lofty status through a combination of ability, opportunity, and utter arbitrary advantage.”13 This is precisely what the two-jar model demonstrates.
Outliers show up in another way. Let's return to Stephen Jay Gould, baseball, and the 1941 season. Not only was that the year that Ted Williams hit .406, it was the year that Joe DiMaggio got a hit in fifty-six straight games. Of the two feats, DiMaggio's streak is considered the more inviolable.14 While no player has broached a .400 batting average since Williams did, George Brett (.390 in 1980) and Rod Carew (.388 in 1977) weren't far off. The closest that anyone has approached to DiMaggio's streak was in 1978, when Pete Rose hit safely in forty-four games, only 80 percent of DiMaggio's record.
“Long streaks are, and must be, a matter of extraordinary luck imposed on great skill,” wrote Gould.15 That's exactly how you generate a long streak with the two jars. Here's a way to think about it: Say you draw once from the jar representing skill and then draw repeatedly from the other. The only way to have a sustained streak of success is to start with a high value for skill and then be lucky enough to pull high numbers from then on to represent your good luck. As Gould emphasizes, “Long hitting streaks happen to the greatest players because their general chance of getting a hit is so much higher than average.”16 For instance, the probability that a .300 hitter gets three hits in a row is 2.7 percent (= .33) while the probability that a .200 hitter gets three hits in a row is 0.8 percent (= .23). Good luck alone doesn't carry the day. While not all great hitters have streaks, all of the records for the longest streaks are held by great hitters. As a testament to this point, the players who have enjoyed streaks of hits in thirty or more consecutive games have a mean batting average of .303, well above the league's long-term average.17
Naturally, this principle applies well beyond baseball. In other sports, as well as the worlds of business and investing, long winning streaks always meld skill and luck. Luck does generate streaks by itself, and it's easy to confuse streaks due solely to luck with streaks that combine skill and luck. But when there are differences in the level of skill in a field, the long winning streaks go to the most skillful players.
Reversion to the Mean and the James-Stein Estimator
Using the two jars also provides a useful way to think about reversion to the mean, the idea that an outcome that is far from the average will be followed by an outcome that is closer to the average. Consider the top four combinations (−3 skill, 4 luck; 3 skill, 0 luck; 0 skill, 4 luck; and 3 skill, 4 luck) which sum to 15. Of the total of 15, skill contributes 3 (−3, 3, 0, 3) and luck contributes 12 (4, 0, 4, 4). Now, let's say you hold on to the numbers representing skill. Your skill remains the same over the course of this exercise. Now you return the numbers representing luck to the jar and draw a new set of numbers. What would