Dealing with Misleading Resesarch

Let me start with an old story — a cartoon that used to hang on my office door.  There is a Congressman sitting at his desk saying "My mail is running 3-1 against the proposal."  The cartoon shows two "stacks" of envelopes — one with three and the other with one.

The realm of probability, sampling, and statistical inference is probably the weakest suit for the supposed Wall Street experts.  Here is a current case of some importance to investors.


Getting a job at Morgan Stanley is exceedingly difficult.  They seem to understand the secret hiring rules that I use at NewArc:

  1. Find the very smartest people.   You cannot coach speed.
  2. Hire those who want to be team players and who create a positive environment.
  3. Rely less on knowledge and more on coach-ability.  I know what they need to know, so the question is whether they are willing to learn.

Morgan Stanley uses non-traditional criteria involving the ability to calculate risk/reward rapidly in game situations.  So how could their team get this research so wrong?

The Research Finding

The key question is whether a peak in the the Economic Cycle Research Institute (ECRI) leading indicators suggests a decline in stocks over the next six months.

The Morgan Stanley study, highlighted by the Pragmatic Capitalist via Abnormal Returns (thanks to both valued sources for highlighting the question) is a bearish scream:

Based on the study, which goes back over 30 years, the equity markets
are down 80% of the time 6 months after a peak in the WLI growth rate.

My Observations

The errors in this approach are manifest.  I expect that regular readers had the radar on high without any help.

  • You should not use percentages  when dealing with a small number of cases.  It implies more data.  Better to say "in five of six cases."
  • This should be a case study analysis.  You only have a few instances and they are all very different.  You do not have enough cases for quantitative analysis.  This is elementary research design in any university curriculum.
  • The frequency analysis does not really support the conclusion.  I looked up the cases, which took only a few minutes.  One is positive, two are neutral and three are negative.  It is not 80% unless you jigger the timing to fit a pre-conceived notion.
  • The ECRI forecasts the economy, not the stock market.  The ECRI is adamant about future economic strength.  It all comes down to whether investors believe that the current market is priced for a brisk rebound.
  • The most recent case is – -quite obviously — without precedent.  The chart below makes this apparent.


This is a classic.  There is no precedent.  You have to reach back 35 years to find anything even close.  How could anyone think that this instance could be forecast from the preceding six cases?


The most obvious conclusion is that one should accord some respect to the ECRI analysis of their own data.

Equally important is the analysis of recessions and post-recessions.  There is an overwhelming desire to draw inferences from slender reeds.

When there is little data, the wise researcher accepts reality.

What is needed is a comparative case study of recessions. If a grad student came to a prof with a proposal with this problem in mind, that would be the only acceptable methodology.

Meanwhile, I encourage young readers to apply for a job at Morgan Stanley!

They need some help.

You may also like


  • Mark Wolfinger April 5, 2010  

    “They need some help.”
    Jeff, If anyone applies to MS with that mindset, I cannot imagine he/she has more than zero chance of being hired.
    If you disagree with what the do, and thus, what they teach, how can you be trainable by their standards?

  • tw April 6, 2010  

    Yes, Morgan Stanley is lame. But has there ever been any evidence to the contrary, evidence that their “insights” make people more money than simply investing in an index fund, over the long term?

  • ChicagoGuy April 7, 2010  

    Hello Jeff,
    Long time reader, first time commenting. By the way, I found you via Barry Ritholtz.
    I’m not sure what will happen next. Got my fingers crossed. Good luck on your bold statements.

  • Jeff Miller April 11, 2010  

    Mark — You are undoubtedly correct. If you want the job, you had better look like one of their people!
    So much for my flippant concluding comments:)

  • Jeff Miller April 11, 2010  

    Hi Chicago —
    Let me know where you are and maybe we can meet some time.
    I do not think that my statements are bold. I think that I am finding the mainstream. It is the popular sentiment that is extreme:)
    Just a thought.