Were the industry's top economists poor practitioners of the art and science of economic forecasting? Or were their disappointing performances simply indicative of how hard it is for anyone to forecast interest rates? I would argue the latter. Indeed, in a nationally televised 2012 ad campaign for Ally Bank, the Nobel Prize winning economist Thomas Sargent was asked if he could tell what certificate of deposit (CD) rates would be two years hence. His simple response was “no.”6
Economists' forecasting lapses are often pounced on by critics who seek to discredit the profession overall. However, the larger question is what makes the job so challenging, and how can we surmount those obstacles successfully. In this chapter, I explain just why it is so difficult to forecast the U.S. economy. None of us can avoid difficult decisions about the future. However, we can arm ourselves with the knowledge and tools that help us make the best possible business and investment choices. That is what this book is designed to do.
Grading Forecasters: How Many Pass?
If we look at studies of forecast accuracy, we see that economic forecasters have one of the toughest assignments in the academic or workplace world. These studies should remind us how difficult the job is; they shouldn't reinforce a poor opinion of forecasters. If we review the research carefully, we'll see that there's much to learn, both from what works and from what hinders success.
Economists at the Federal Reserve Bank of Cleveland studied the 1983 to 2005 performance of about 75 professional forecasters who participated in the Federal Reserve Bank of Philadelphia's Livingston forecaster survey.7 We examine their year-ahead forecasts of growth rates for real (inflation-adjusted) gross domestic product (GDP) and the consumer price index (CPI). (See Table 1.1.)
Table 1.1 Accuracy of the Year-Ahead Median Economists' Forecasts, 1983–2005
* Assigned by the author.
Source: Michael F. Bryan and Linsey Molloy, “Mirror, Mirror, Who's the Best Forecaster of Them All?” Federal Reserve Bank of Cleveland, Economic Commentary, March 15, 2007.
If being very accurate is judged as being within half a percentage point of the actual outcome, only around 30 percent of GDP growth forecasts met this test. By the same grading criteria, approximately 39 percent were very accurate in projecting year-ahead CPI inflation. We give these forecasters an “A.” If we award “Bs” for being between one-half and one percentage point of reality, that grade was earned by almost 22 percent of the GDP growth forecasts and just over 30 percent of the CPI inflation projections. Thus, only around half the surveyed forecasters earned the top two grades for their year-ahead real GDP growth outlooks, although almost 7 in 10 earned those grades for their predictions of CPI inflation. (We should note that CPI is less volatile – and thus easier to predict – than real GDP growth.)
Is our grading too tough? Probably not. Consider that real GDP growth over 1983 to 2005 was 3.4 percent. A one-half percent miss was thus plus or minus 15 percent of reality. Misses between one-half and one percent could be off from reality by as much as 29 percent. For a business, sales forecast misses of 25 percent or more are likely to be viewed as problematic.
With that in mind, our “Cs” are for the just more than 17 percent of growth forecasts that missed actual growth by between 1 percent and 1.5 percent, and for the 22 percent of inflation forecasts that missed by the same amount. The remaining 30 percent of forecasters – those whose forecasts fell below our C grade – did not necessarily flunk out, though. The job security of professional economists depends on more than their forecasting prowess – a point that we discuss later.
The CPI inflation part of the test, as we have seen, was not quite as difficult. Throughout 1983 to 2005, the CPI rose at a 3.1 percent annual rate. Thirty-nine percent of the forecasts were within half a percent of reality – as much as a 16 percent miss. Another 30 percent of them earned a B, with misses between 0.5 and 1 percent of the actual outcome, or within 16 to 32 percent of reality. Still, 30 percent of the forecasters did no better than a C.
In forecasting, as in investments, one good year hardly guarantees success in the next. (See Table 1.2.) According to the study, the probabilities of outperforming the median real GDP forecast two years in a row were around 49 percent. The likelihood of a forecaster outperforming the median real GDP forecast for five straight years was 28 percent. For CPI inflation forecasts, there was a 47 percent probability of successive outperformances and a 35 percent probability of beating the median consensus forecast in five consecutive years.
Table 1.2 Probability of Repeating as a Good Forecaster
* Proportion expected assuming random chance.
Source: Michael F. Bryan and Linsey Molloy, “Mirror, Mirror, Who's the Best Forecaster of Them All?” Federal Reserve Bank of Cleveland, Economic Commentary, March 15, 2007.
Similar results have been reported by Laster, Bennett, and In Sun Geoum in a study of the accuracy of real GDP forecasts by economists polled in the Blue Chip Economic Indicators– a widely followed survey of professional forecasters.8 In the 1977 to 1986 period, which included what was until then the deepest postwar recession, only 4 of 38 forecasters beat the consensus. However, in the subsequent 1987 to 1995 period, which included just one mild recession, 10 of 38 forecasters outperformed the consensus. Interestingly, none of the forecasters who outperformed the consensus in the first period were able to do so in the second!
Perhaps even more important than accurately forecasting economic growth rates is the ability to forecast “yes” or “no” on the likelihood of a major event, such as a recession. The Great Recession of 2008 to 2009 officially began in the United States in January of 2008. By then, the unemployment rate had risen from 4.4 percent in May of 2007 to 5.0 percent in December, and economists polled by the Wall Street Journal in January foresaw, on average, a 42 percent chance of recession. (See Figure 1.1.) Three months earlier, the consensus probability had been 34 percent. And it wasn't until we were three months into the recession that the consensus assessed its probability at more than 50 percent.
Figure 1.1 Unemployment and Consensus Recession Probabilities Heading into the Great Recession of 2008–2009
Source: Bureau of Labor Statistics, The Wall Street Journal.Note: Shaded area represents the recession.
The story was much the same in the United Kingdom (UK). By June of 2008 the recession there had already begun. Despite this, none of the two-dozen economists polled by Reuters at that time believed a recession would occur at any point in 2008 to 2009.9
In some instances, judging forecasters by how close they came to a target might be an unnecessarily stringent test. In the bond market, for example, just getting the future direction of rates correct is important for investors; but that can be a tall order, especially in volatile market conditions. Also, those who forecast business condition variables, such as GDP, can await numerous data revisions (to be discussed in Chapter 5) to see if the updated information is closer to their forecasts. Interest-rate outcomes, however, are not revised, thereby denying rate