3. Statistical crosscurrents make it hard to find safe footing.
Even if the past and present are clear, divining the future remains challenging when potential causal variables (e.g., the money supply and the Federal purchases of goods and services) are headed in opposite directions. However, successful and influential forecasters must avoid being hapless “two-handed economists” (i.e., “on the one hand, but on the other hand”).
Moreover, one's statistical coursework at the college and graduate level does not necessarily solve the problem of what matters most when signals diverge. Yes, there are multiple regression software packages readily available that can crank out estimated regression (i.e., response) coefficients for independent causal variables. But, alas, even the more advanced statistical courses and textbooks have yet to satisfactorily surmount the multicollinearity problem. That is when two highly correlated independent variables “compete” to claim historical credit for explaining dependent variables that must be forecast. As a professional forecaster, I have not solved this problem but have been coping with it almost every day for decades. As we proceed, you will find some helpful tips on dealing with this challenge.
4. Behavioral sciences are inevitably limited.
There have been quantum leaps in the science of public opinion polling since the fiasco of 1948, when President Truman's reelection stunned pollsters. Nevertheless, there continue to be plenty of surprises (“upsets”) on election night. Are there innate limits to humans' ability to understand and predict the behavior of other humans? That was what the well-known conservative economist Henry Hazlitt observed in reaction to all of the hand wringing about “scientific polling” in the aftermath of the 1948 debacle. Writing in the November 22, 1948, issue of Newsweek, Hazlitt noted: “The economic future, like the political future, will be determined by future human behavior and decisions. That is why it is uncertain. And in spite of the enormous and constantly growing literature on business cycles, business forecasting will never, any more than opinion polls, become an exact science.”13
In other words, forecast success or failure can reflect “what we don't know that we don't know” (generalized uncertainty) more than “what we know” (risk).
5. The most important determinants may not be measureable.
Statistics are all about measurement. But what if you cannot measure what matters? Statisticians often approach this stumbling block with a dummy variable. It is assigned a zero or one in each examined historical period (year, quarter, month, or week) according to whether the statistician believes that the unmeasurable variable was active or dormant in that period. (For example, when explaining U.S. inflation history with a regression model, a dummy variable might be used to identify periods when there were price controls.) If the dummy variable in an estimated multiple regression equation achieves statistical significance, the statistician can then claim that it reflects the influence of the unmeasured, hypothesized causal factor.
The problem, though, is that a statistically significant dummy variable can be credited for anything that cannot be otherwise accounted for. The label attached to the dummy variable may not be a true causal factor useful in forecasting. In other words, there can be a naming contest for a dummy variable that is statistically sweeping up what other variables cannot explain. There are some common sense approaches to addressing this problem, and we discuss them later.
6. There can be conflicts between the goal of accuracy and the goal of pleasing a forecaster's everyday workplace environment.
Many of the most publicly visible and influential forecasters – especially securities analysts and investment bank economists – have job-related considerations that can influence their advice about the future. It is ironic that financial analysts and economists whose good work has earned them national recognition can find pressures at the top that complicate their ability to give good advice once the internal and external audience enlarges.
Many Wall Street economists, for instance, are employed by fixed-income or currency trading desks. Huge amounts of their firms' and their clients' money are positioned before key economic statistics are reported. This knowledge might understandably make a forecaster reluctant to go against the consensus. And, as we discuss shortly, there can be other work-related pressures not to go against the grain as well.
Are trading desks' economists' forecasts sometimes made to assist their employers' business?
It is hard, if not impossible, to gauge how much and how frequently forecasts are conditioned by an employer's business interests. However, it can be observed that certain types of behavior are consistent with the hypothesis that forecasts are being affected in this manner. For instance, the economist Takatoshi Ito at the University of Tokyo has authored research suggesting that foreign exchange rate projections are systematically biased toward scenarios that would benefit the forecaster's employer. He has attached the label “wishful expectations” to such forecasts.14
What is the effect of the sell-side working environment on stock analysts' performance?
In order to be successful, sell-side securities analysts at brokerage houses and investment banks must, in addition to performing their analytical research, spend time and effort marketing their research to their firms' clients. In buy-side organizations, such as pension funds, mutual funds, and hedge funds, analysts generally do not have these marketing responsibilities. Do the two different work environments make a difference in performance? The evidence is inconclusive.
For instance, one study funded by the Division of Research at the Harvard Business School examined the July 1997 to December 2004 period and reached the following conclusions: “Sell-side firm analysts make more optimistic and less accurate earnings forecasts than their buy-side counterparts. In addition, abnormal returns from investing in their Strong Buy/Buy recommendations are negative and under-perform comparable sell-side recommendations.”15
There is a wide range of performance results within the sell-side analyst universe. For example, one study concluded that sell-side securities analysts ranked well by buy-side users of sell-side research out-performed lesser ranked sell-side analysts.16 (Note: This study, which was sponsored by the William E. Simon Graduate School of Business Administration, reviewed performance results from 1991 to 2000.)
How does media exposure affect forecasters?
To see how the working environment can affect the quality of advice, look at Wall Street's emphasis on “instant analysis.” Wall Street economists often devote considerable time and care to preparing economic-indicator forecasts. However, within seconds – literally, seconds – after data are reported at the normal 8:30 a. m. time, economists are called on to determine the implications of an economics report and announce them to clients.
Investment banks and trading firms want their analysts to offer good advice. But they also want publicity. They're happy to offer their analysts to the cameras for the instant analysis prized by the media. The awareness that a huge national television audience is watching and will know if they err can be stressful to the generally studious and usually thorough persons often attracted to the field of economics. Keep this in mind when deciding whether the televised advice of an investment bank analyst is a useful input for decision making. (Note: Securities firms in the current, more regulation-conscious decade generally scrutinize analysts' published reports, which should make the reports more reliable than televised sound bites.)
7. Audiences