Predictions versus Forecasts

A favorite pastime of market pundits is making sport of those who do forecasting, especially if models are involved.  I am continually amazed at the delight people take in criticizing things where they have no information.  The ongoing mission at "A Dash" is to discover and exploit such errors.

Background Example

Let us suppose that we have a city in an earthquake zone.  There is a fault line.  Our job is to predict something mundane, like revenue over a toll bridge.  We may use various factors to predict traffic and revenue as the basis for the forecast.  Meanwhile, we all know that there is earthquake potential.  If and when the earthquake comes, the revenue forecast will be seriously wrong.

So we could make a prediction:  There will be an earthquake in the next ten years.

Or we can make a forecast:  Here is the expected bridge traffic next year.

Please note that the forecaster is usually correct by ignoring the earthquake.  If he decides to include it, he can be wrong in one of two ways:

• He can deduct something from each year's forecast, making them all incorrect by 10% or so; or
• He can guess the year of the quake. He will be correct one year in ten and wrong in the other years.

Let me consider how people mistakenly confuse the two approaches in analyzing the stock market.

A Conversation with George

(This is a composite conversation with real people, but not a specific one.  It is accurate, in that it reflects the feelings of many, but it is not an actual dialog.)

I got a call from George, an intelligent market observer who reads a lot.

G:  What do you think about the market?  I am getting worried after the big move.

J:  Are you all-in?

G:  Are you kidding?  I am not in at all.  I just heard a prediction that the S&P 500 is going to 400.

J:  Who said that?

G:  Someone with a system.  He said to ignore predicted earnings because analysts are biased.  He uses only actual known earnings.  Everything else is a lie.  His history went back to the time of Paul Revere or something — great research.  He also pointed out that we had to use a "trough multiple" — or — just a second — that was what we used last year.  Now we should use the peak multiple, but they are actually both the same.  You multiply $50 in S&P earnings by 8, and that is what the market should be.  Do you know what the P/E ratio was in 1840?

J:  Uh…..no.  May we return to the 21st century?

G:  I thought you were a scholar.  But OK.

J:  It has taken 18 months to regain the pre-Lehman levels.  The selling in late '08 and early '09 were predictions of another depression and a total failure of the Obama Administration.  It has taken this long to return to a semblance of sanity.

G:  So what?  There is always a correction after a big move.  We are going lower.  Much, much lower.

J:  Markets do not go straight up, but we are just getting back to a starting point, something that I described as an "initial target" at the start of 2009.  Nearly every indicator, except employment, is better than it was then.  Markets are about earnings, and earnings are showing great strength.

G:  I think that the market will move sideways for many years — or maybe sell off.

J:  Why?

G:  The economy cannot  grow without the consumer.  I read that consumers are 70% of the economy and that they are tapped out, spent up, unemployed, and leveraged to the hilt.  the market is going nowhere without the consumer.

J:  The mainstream economic forecasts…..

G:  Forecasts!  Don't give me that!  Those economists know nothing!  They know nothing!  They did not see it coming. They use models.  That is just fantasyland.  Everyone knows that models have built-in error.  Economists know nothing.  Obama knows nothing.  Congress knows less than nothing.  These are the guys who got us into trouble and I am not going to believe them.

J:  OK.  Let us put aside the political and methodological rant.

G:  It is just the truth.  We will be lucky to stay sideways in this market.  I have actually been checking out the price of farmland.

J:  How is your ammo?

G:  Locked and loaded.

J:  Returning to the market, you just told me what you expect.  How did you arrive at those conclusions?

G:  I read some great sources, including many who predicted the crash.  They explain that the forecasters are all wrong.  None of the models worked.  The guys I read all saw it coming.

J:  Did any of them change their opinions at any point?  After the fall of Lehman?  After the government policy actions?  After improving economic data?

G:  Change?  No way!!  They got it right.  Lehman and that other stuff made no difference.

………..

The Dilemma

As an investment advisor, how do you help someone with this mind set?   There are several very fundamental problems.  Let me take them one at a time.

Predictions are often very general and completely ignore the time frame.  We started seeing recession forecasts in 2004 or so.  Keeping in mind that a recession happens (on average) every five years, what should be the shelf life for these predictions?  Those who grew impatient started arguing their own definitions for what constituted a recession.

Predictions involve modeling.  The difference is that for many people the models are poorly specified, based on little information, and cannot be tested.  Consider George's assertions.  He actually made a commonly-accepted statement that has hidden modeling and assumptions.

• Economy and the consumer.  Those who cite the 70% statistic do some while ignoring a long list of assumptions.  Most consumer spending holds up even in recession.  Not everyone is affected, and even those who are have various means of support.  There are also new population entrants.  Finally, there are times when business takes the lead, and we might be in such a cycle.
• Economy and the stock market.  Those who point to economic concerns think that the market is a GDP future.  This is quite clearly not correct.  Many companies can and do thrive even in adverse economic circumstances.

My point is not that the predictions from George's pundits will be wrong.  The point is that the reasoning is very sloppy.

Anyone making a prediction is using a model!

Those who make predictions while criticizing modelers simply lack the requisite training.  A professional modeler identifies risks and possible error.  Every modeler understands that the result is a simplification of reality.  Good modelers test results against reality and suggest error ranges.

Compare this highly professional approach with the "laundry list" method of many pundits who have no formal training in models.  They cite lists of "headwinds" and make predictions without time frames.  Those who rail against models and the "folly of forecasting" while still making predictions are still doing modeling, but doing it very poorly.

There will be an earthquake some day…..

Investment Conclusion

Investors should be paying attention to the analysis of the causes of the financial crisis.  There is a complex story including many factors.  Understanding what happened could be important to making the right investment choices.

So how should I explain this to George, whose mind is made up?