How Good Are Economic Forecasters?
Economic forecasters make an easy target! They seem
dull and boring, their work is complicated, and they interfere with the
preferred method of the punditry: Speculation and anecdotes.
If forecasters are no good, we are all free to ignore them and their
data. Each person’s idea is as good as anyone else’s in the data-free
world. Thanks to Crossing
Wall Street for covering a research note by Martin Fridson dealing with this
interesting question.
Fridson, is a prolific and
respected financial author, with strong educational and professional
credentials. His writing on this subject will get a lot of attention, no
doubt. The media and pundits will love it, since it tells them what they
want to hear.
Our conclusions?
- We are confident that the evidence reported does not support the principal conclusion, that the market is a better forecaster than the economists.
- We suspect that Mr. Fridson has never done any quantitative forecasting.
- We believe that investors who follow his advice will find their choice an expensive one.
- We recommend that investors ignore his little research note and instead read one of Mr. Fridson’s several other fine, instructive, and entertaining books emphasizing market history.
For the rationale ….read on.
The Right Way and the Wall Street Way
There is a way to conduct professional research. Briefly put, one formulates the question or hypothesis, usually linking it to some broader issues of significance. One then develops a research design that will address that question. This involves the ability to find, to collect, or to create the required data. (If you do not have enough of the right data, you do not have a project.) One establishes criteria or statistical methods to determine how the question will be resolved. Finally, one collects the data and makes the appropriate tests and comparisons.
By contrast, Wall Street researchers find a data series and make a chart or a list of examples. They rarely ask if they have the right data or enough cases of differing types. They then invoke software to draw a chart and offer an opinion about what they see. At this point the financial media and bloggers uncritically transmit the findings to everyone else. The dubious results become part of the conventional wisdom.
Fridson’s Choice
We were disappointed to see Fridson follow the latter approach. Apparently, he wants to draw a conclusion about the usefulness of economic forecasts for investors and policymakers. (We know this only from reading his conclusion, since the research question is never really stated). He warns investors (in his conclusion) that it is "highly speculative" to bet that economic forecasts are better than the prevailing prices of "economically sensitive stocks."
OK, simple enough. Get some data about economists’ forecasts. Figure out a method for inferring economic forecasts from the market — something like low PE ratios for cyclical stocks should work. Get a lot of data for each method, making sure that you go through a few significant eonomic turns and business cycles. Compare the two approaches to see which does better.
Fridson does almost none of this. He has a data series covering four years (only!!) of GDP forecasts by economists, and he makes a chart of that data. Then he gives us his opnion that the resulting forecasts are of no use. The data cover a single period of above-trend growth including no business cycles and no recessions. He makes no effort to compare the forecasts with those implied by the market.
There is no reason to believe Fridson’s conclusions, because he makes no effort to prove them!
The Real Subject of the Fridson Article
So what is the point of the research? Let’s try to look at the data through Fridson’s eyes.
Here is one of the tables described in the article. It compares one year advance predictions by economists with actual year-over-year GDP change by quarter.
(click to enlarge)
Here is what Fridson sees in his data:
"Exhibit 2 shows that during 2001-2006, the year-ahead forecast hardly
varied from one year to the next. The median prediction was in the range of
3.1% to 4.0% in every single quarter. Perhaps not coincidentally, the actual
quarterly GDP increase over the past 25
years (1981-2005) averaged 3.14%. The forecasters, in aggregate, perennially
thought that one year hence, business conditions would be just about average.
In reality however, actual GDP gains gyrated
between 0.2% and 7.5%. The forecasters’ nearly inert consensus was all but
worthless."
This summary is misleading at best.
First, the economists’ forecasts were actually quite good! During this brief period of time the economy showed strength above the
long-term trend of 3.14%, and this strength was predicted by the
consensus forecast. That is the single most important element of the chart. Since the stock market has gone through 2 1/2 years of multiple compression, with many analysts calling the top of the cycle since 2004, I doubt that a prediction series based upon those prices would have done as well.
Second, (and this is the part that suggests a lack of forecasting background by the research team) it is completely normal for the prediction series to have much less volatility than the actual data. This is a characteristic of any quantitative model — at
least any of them that have value. The process of model-building
involves finding relevant data, smoothing series to get the most
effective signal-to-noise ratios, and similar processes. Models that
"swing for the fences" will usually be wrong, and wrong by a lot!
In fifty years of
quantitative forecasting (yes, I started by predicting the Dodgers and
Yankees to win pennants in 1956) I have seen thousands of models, but it only takes a little experience to know this basic fact. You can learn it in five minutes by trying this simple test. Pick up the newspaper and look at the NFL football lines for today’s games. Then look at the final scores. Did the pointspreads accurately reflect the biggest swings? Do you think that the oddsmakers’ lines lack predictive value because of this?
I hope you are answering these rhetorical questions accurately or "They will send a limo for you!" Predictions vary less than actual results, especially in time series data.
Yet this lack of variation in predictions is the essence of Mr. Fridson’s criticism when he tells you to ignore the economic predictions. His conclusions would be more persuasive if he offerred something better.
Finally, why is it important to predict the year-over-year GDP on a quarterly basis? That is, why do we care about how GDP growth from Q3 2002 to Q2 2003 versus the calendar year starting one quarter later? It is an arbitrary and misleading way of fragmenting the data. Thinking about the history of this period may help to explain this.
The first big deviation (2002) encompassed a period where the pattern of business activity (but not the trend) was interrupted by anticipation of a Longshoremen’s strike which added to Q3 2002 at the expense of Q4. There was also a shifting of economic activity when the nation cautiously prepared for the Iraq war followed by a rebound when the war seemed to be ending quickly. This behavior was obvious to anyone listening to CEO interviews or conference calls.
In both cases economic activity was shifted from one quarter to another. Even if one could devise a method of knowing this in July of 2001, of what use would it be?
The second big deviation (2005) was Hurricane Katrina followed by a rebuilding rebound. This was certainly not predictable a year in advance. Even if one did know that growth would be shifted by a quarter, of what significance would that be?
In other words, we are not as interested in quarter-by-quarter deviations as we are in determining economic turning points. We cannot study those in Fridson’s data because he does not include any! It is almost as if the choice of data to study was made for a different purpose.
Political Agenda?
We hope that anyone reading this has been convinced to reject Mr. Fridson’s advice about economic predictions. At "A Dash" we are trying to educate and inform. We hope that our readers gain a financial advantage. We do not have a political agenda. We get worried when resesarch mixes policy and investment advice. That is why his last recommendation makes us particularly uneasy.
Consistent with the theme of his most recent book, about keeping government out of the markets, Mr. Fridson writes as follows:
"As for government policymakers, the message is to forget about trying to
control short-run economic performance. Given the lagged impact of fiscal or
monetary intervention, deciding whether stimulus or restraint is needed depends
on knowing where GDP will be a few quarters
down the line. That isn’t something economists have shown they can reliably
predict. A more appropriate mission for government policy is to refrain from
meddling that ultimately undermines confidence among business and consumers."