Why the average investor is bamboozled by recession forecasts

I have a problem.  I cannot teach my research methods course in one short article – -not even the undergraduate version.

Since the topic is so important I want to do it right by conveying the most important features.  While readers of this article may not be able to write a critique of various research findings, you might develop a strong instinct for what is right.

Background

I have served as a referee for scholarly journals, as a methods prof at top schools, and most significantly, as a gatekeeper judging investment research.  You might be surprised to learn that much of the work at big-time firms would not pass muster by these standards.  The criteria for marketing and sales are different from peer review and actually putting your money on the line.

How to spot suspicious results

An expert can spot bogus results.  Suppose you assign a group of students to flip a coin 200 times and write down the results.  Some of them actually do it, while others fake it.  I could spot the fakes in a heartbeat, and so could you with a little practice.  Most people do not understand what random data looks like, so they do not know how to fake it.  They do not understand that "streaks" happen.  By the way, the IRS knows — no problem for us honest folks!  Read here to learn more.

In the investment world, this is common with trading systems.  As the resident prof and gatekeeper for my trading firm in the late '80s, we got to see plenty of trading systems.  By way of background, the 1987 crash was preceded by extremely oversold conditions, a situation that many thought was right to buy.  Put premium was at unprecedented levels, and put selling was prevalent.  Even the strongest and most experienced traders sold a few puts into that fateful Friday close before the crash.  On Monday,  October 19th, 1987 there was no escape from the worst day in market history.

Very smart, very successful traders had spotted a buy signal that had been profitable for many years.  When people came around to sell trading systems, how many of them recommended a "buy" on that day?  If you are guessing "ZERO" then you are correct.  Even though the "syndrome" of successful indicators would probably resume working for another 20 years, no system developer would admit it.  It was almost as if they had a specific binary variable — exclude any signal on a Friday in October of odd numbered years.  I am intentionally exaggerating this, of course, but it is essential.  Anyone with a computer can find something plausible that has the same effect and is more difficult to spot.  Some of the current bogus research uses this very approach — too many variables for too few events.

Examples of Strong Research

Great research starts with a hypothesis.

My favorite professor in grad school tricked us, something that I chronicled in this article.  If you take the time to read it, you will see the error of modern "experts" who start with data instead of following the scientific method.

This same prof had a great study of innovation in public health agencies.  He discovered that agencies that had great motivation to innovate but lacked money produced little innovation.  Agencies that had plenty of bucks but a leader who was a dinosaur also did not innovate.  It took both.  He wrote a great analysis of statistical interaction.  Note the relevance of the theory, his brilliant starting point.  He did not collect some data and then define a "syndrome."

My dad solved many problems the same way — fuel, oxygen, ignition.  It was amazing how many problems he solved this way, as I wrote in this article.  The story is amusing, but Dad had a solid lesson, which included some thoughts about respect and learning something from everyone.

Implications for Current Recession Forecasts

There is plenty of room for debate.  In my careful, months-long survey of recession forecasting I concluded that there were questions about each method.  There is a healthy discussion on this subject.  I hope that I have helped to encourage this, and I will continue to contribute.  I hope readers will understand that this is one of many important themes.  I see the concern as important but not urgent, so I am going at a careful pace.

Let us split the candidates into two groups, those that have a long-term record, and those who only started with the last recession — or even later.

Long-term record

I still invite new nominations on this, but I think that Bob Dieli, the Leading Economic Indicators, and the ECRI are the only sources with such a record.

The LEI is a moving target.  The early roots were promising, and for years it was maintained by the government through the Commerce Department.  Some authors who wish to disparage the current findings still incorrectly refer to it as a government indicator.  A more telling criticism is that the components keep changing, with a complete repainting of the data history.  If you really want to know the record of this indicator, you need an "as reported" history.

The ECRI remains a mystery, but one that is closer to a solution every day.  A worldwide team of very bright and talented people have succeeded in reverse engineering the components of the index and also the mysterious translation to a growth index with a complex calculation.  I do not want to spoil the emerging story, but it  will not take long.  There are several key questions:

  1. Is the published history based on the current index?  If not, doesn't this seem like false advertising?  If there are changes, what were they?  (I do not ask this lightly.  The team can replicate the old history almost perfectly, but not the current recession call — the very thing we are all wondering about).
  2. Does the method rely heavily upon commodity prices?  Didn't these spike in the first part of 2011 for reasons that many attribute to speculation, the Fed or other non-fundamental factors?  (Don't get me started on this!)  If so, would not a reversion to regular growth levels just be "normal" and not an indication of economic collapse?
  3. Why did your story switch so dramatically?  One moment it was the idea that an era of slow growth made us more vulnerable.  In a heartbeat it switched to a 100% recession call with no effective policies for government leaders.  What happened?  Did the methods change?

Bob Dieli has done the best over many decades.  Like every model developer,  Bob is always questioning both his approach and his conclusions.  As a reviewer and critic, I am doing the same.  Unlike the ECRI, the Dieli method is open to review, and I will take this up in the next article.

Here is the key concept:  Most recession forecasters mistakenly look for weakness.  That tells you that the current economy is performing below trend, but it is not a signal of a new recession.

Bob's method looks for cycle peaks (and troughs), which is how the NBER defines recessions (and the end of recessions).  If you do not understand this, go back to the first article in this series.

The short-term candidates

There are several recession forecasters who do not have a long-term record in real time.  Essentially, they are doing research that tries to explain the past, sometimes without following a scientific method.  Since the statistical procedure and commentary are fancy, it may seem persuasive.

This is the illusion that comes from using too many variables on too few cases.  In fact, the fewer variables the better!  Especially when there are only a few cases to predict, using a long laundry list of binary indicators is a red flag!  You will overfit (over-explain) the few past cases and be forced to keep revising your indicators over time.

Hint for Careful Readers

Those who want to avoid being bamboozled can ask a simple question:

When someone tells me about a syndrome or Aunt Gertrude's symptoms, have they done the following:

  1. Provided a hypothesis, stated in advance;
  2. Explained the reason for the proposed statistical interaction;
  3. Demonstrated a real-time record of recession forecasting; and
  4. Maintained the same exact criteria over time (no fudging, changing, suggesting of Plan B or exceptions).

If not, taking up my old roles, I would not publish the article nor would I invest in the system.  It is sad that so many individual investors think that something that looks perfect is the best choice.  If only they had seen more such examples, they would have better instincts.

I hope this article will help.

You may also like

19 comments

  • Paul January 18, 2012  

    This article REALLY helps. I have learned a lot from these articles Professor!

  • David January 18, 2012  

    “Here is the key concept: Most recession forecasters mistakenly look for weakness. That tells you that the current economy is performing below trend, but it is not a signal of a new recession.”
    I couldn’t have put it better myself. On top of that, they also tend to elaborate upon new data in light of the previous expansion. I feel that to do so ignores the dynamic nature of the economy, markets, demographics, and culture.
    Anyway, thank you for the post.

  • David January 19, 2012  

    Err, what I meant to say was that many forecasters examine the current expansion with an older mental framework which worked well in understanding recent ones. To me, that isn’t always justified. When they do so however, it seems that this can cause significant missteps when they make judgments. My mistake if there was any confusion.

  • RB January 19, 2012  

    The ECRI analysis is interesting – thanks for pointing it out. There might just be something useful there in combination with a simple 200 day moving average monthly prices based system such as Meb Faber’s.

  • RB January 19, 2012  

    Also, I calculated the ECRI growth index from the WLI as follows:
    Growth index (ratio) = (SMA4/SMA52)^(52/28)
    based on the moving averages, essentially annualizing the change based on its center 6.5months ago.
    The growth index (%) is realized from above 100*(Growth_index_ratio -1).

  • RB January 19, 2012  

    BTW, SMA4=most recent 4-week moving average
    SMA52=previous 52-week moving average prior to most recent 4-weeks.
    The accuracy is better than the article reports using a 37-week average and is in fact the method used by ECRI, for which I can find some supportive links, if I feel the motivation to.

  • WallStreet_Rant January 19, 2012  

    “Suppose you assign a group of students to flip a coin 200 times and write down the results. Some of them actually do it, while others fake it. I could spot the fakes in a heartbeat, and so could you with a little practice. Most people do not understand what random data looks like, so they do not know how to fake it. They do not understand that “streaks” happen.”
    But maybe you are too confident that you could spot the fakes based on what you “think” randomness really looks like. Seems you think a lack of streaks implies faking. But reality is, in true randomness there is no way to be certain you spotted a fake….

  • oldprof January 19, 2012  

    RB — Thanks for sharing your analysis on this. Any links you could provide would be most welcome.
    Jeff

  • oldprof January 19, 2012  

    WallStreet_Rant — The statistical properties of such random distributions have been extensively studied and are well known.
    What do you think the % chance is that a 200 flips will have no streak of 6? A streak of 7?
    This is not a matter of opinion, but one of knowledge.
    Jeff

  • RB January 19, 2012  

    Jeff,
    There is this link which states:
    The growth rate is based on a four-week moving average of the weekly leading index level, compared with the previous year’s moving average.
    ECRI has explained how they annualize the numbers i.e. exponent of (52/26.5) for weekly series. I extrapolated this to the expression I gave which matches the growth index very well. In fact, I shared this with Doug Short who asked for my comments on this article:
    http://findarticles.com/p/articles/mi_m1094/is_4_34/ai_56973863/
    which further illustrates how the exponent is calculated. I think he didn’t buy my explanation because he has a simpler version on his ECRI posts.

  • Bud January 21, 2012  

    Seriously, if you can spot a fake in an heartbeat how come ECRI is still a mystery? When random data is reduced to a moving average isn’t the randomness of the data altered? I don’t see any conviction in your analysis…it provides more questions then answers…???

  • oldprof January 21, 2012  

    Bud — I clearly did not state that the ECRI data is faked or a random series. If you follow the links to the research that I have described you will see what is known about their approach.
    Let’s try a different approach. Are you a football fan? Suppose someone came to you with a system that had correctly predicted every Super Bowl — every one. He created the system last week. Meanwhile, you have a friend you have known for decades who called 70% of the games in real time. Which one would you expect to be right this year?
    As to answers — I have two or three more articles in this series. Maybe I’ll get a little closer, but it is a topic where we’ll never have all the answers. You should share my suspicion of anyone who thinks he does.
    Jeff

  • HenryE January 22, 2012  

    Dr. Miller,
    I think you’ve done an excellent job highlighting the weak analyses of many economic/maket commentators. You also generally defend the competence and professionalism of members of the Fed and government officials.
    A few days ago, I read extracts from transcripts of the Fed minutes from 2006 (since apparantly the Fed releases these with a 5 year lag). These showed a high degree of ignorence and complacency about the size of the bubble in the housing market, the lengths to which people were substituting debt for income, and the severe dislocations that would take place when that particular ponzi game stopped. Obviously the extracts were picked to show the Fed members in the worst light, but these were their words and they give a strong impression of not understanding that a massive proportions was about to go pop.
    I’m sure you read the same transcripts. Would you not say that the Fed, despite all the brilliant and educated people at its disposal, was just as clueless in the mid-2000’s as the people who have been calling for another crash ever since March 2009?

  • oldprof January 22, 2012  

    HenryE — This is a good question, and you are right about my general viewpoint. I have been thinking about doing something on the topic. Usually I download all of the new year’s transcripts, making them easier to search and to analyze.
    My basic viewpoint is summarized in this piece about Bernanke. https://www.dashofinsight.com/a_dash_of_insight/2011/04/a-new-viewpoint-on-bernanke.html
    Very good analysts — hardly clueless — looked at the subprime problem and calculated the total possible effect. This is why people thought they had a handle on the problem.
    They did not know that the total amount of synthetic mortgage obligations was much larger than the actual market. There was no reporting or transparency. We could say that nearly everyone was clueless in the sense that there was no good contemporaneous information on the size of this derivative market.
    Some of the banks knew or suspected, as did the heroes written up by Michael Lewis in The Big Short. No one believed them at the time.
    The effect of this has wrecked the housing market in many areas where there was never a bubble to begin with.
    Anyway, that is my brief and basic answer to your very good question. I know it requires more evidence and explanation, but you have a reasonable attitude toward the issue.
    Jeff

  • HenryE January 22, 2012  

    Thank you for pointing me to that previous posting. I’ve actually been reading a lot of your previous writings in the archives. I wish I had read your articles regarding FAS 157 before March 2009.

  • oldprof January 22, 2012  

    HenryE — You might also like the one where I reviewed The Big Short — https://www.dashofinsight.com/a_dash_of_insight/2010/08/book-review-the-big-short-by-michael-lewis.html
    This is really crucial for drawing the right lesson from the crisis.
    Concerning 2009 — the forward P/E ratio is now lower than it was in May of 2009. I’ll be taking up that theme in a couple of weeks:)
    Thanks again for joining in.
    Jeff

  • V K Chandy January 22, 2012  

    I would hesitate to make the assertion, “…I can usually spot the faker…”. Don’t you think prof, that it would be more correct to say, “In a sample of 100 sets of 200-coin toss sequences, I can say with a great deal of confidence that x% of this 100-set sample have faked results, based on deviations from the norm. however, it is impossible to say who has faked the result and who has not.” (As an aside, an expert who knows statistics can insert just the right number of 6 or 7 heads/tails in a row sequences…i.e. he is a faker, but a truly innocent participant who did not fake the result may in fact have ended up with only 4 or 5 H/T sequences purely randomly..i.e. she did not fake the result…)

  • oldprof January 22, 2012  

    VK – -No, you are incorrect. Out of the group of fakers, I can spot them with high assurance. This is a long-time experiment, as the link showed. It is not a matter of opinion, but one of fact.
    If you want to tell me the odds of a streak of 7 in a row, we can continue the discussion from there. Until and unless you know this, you are just doing a seat-of-the pants opinion on something that is a matter of statistics.
    Meanwhile, I am amazed that anyone is questioning this point. It just goes to show why we can make so much money in the markets. People are willing to believe contrived data and unable to recognize those who really understand a specific problem.
    Anyway, why don’t you provide the x% of fakes in the sample, and then we can revisit my assertion, and that of the Professor I cited in the link —
    Jeff

  • Eric Kennedy January 27, 2012  

    Jeff, when I first saw ECRI’s stuff posted by Anirvan Banerji on TheStreet.com in 2000, the recession calls were based off a persistent decline in the US Long Leading Index (US LLI). The WLI is a shorter leading index (that’s why it’s not called the Weekly Long Leading Index). As you can infer from this article
    http://www.thestreet.com/story/10001661/1.html
    the LLI is a monthly series, while the WLI is a noisy weekly series.
    ECRI doesn’t publish USLLI data publicly, which tells you that they value that more than WLI. However, sometimes it is distributed in chart form, like this:
    http://www.ritholtz.com/blog/2010/07/u-s-long-leading-index/
    ECRI didn’t call a recession in 2010 even when WLI growth went negative because USLLI had not had a pronounced, pervasive and persistent decline until early 2011.
    Given today’s very light nominal GDP data and the recessions in Europe and slowdowns in Asia, it looks like ECRI will be right. In 1990 and 2000, they were 5-6 months ahead of a recession, while they were a bit late saying that policy actions could have helped in January 2008.