Developing and Evaluating Trading Systems

Improved technology, more power.  We would expect this to be good.

In fact, more power can enable us to do exactly the wrong thing.

This happens all of the time with the world’s most powerful computer, the human mind.  A year ago we reviewed analysts who thought the market looked like a replay of the 1987 crash.  This type of analysis crops up all of the time, often using old charts as evidence.  With the power to search among thousands of choices, picking the time frame, and adjusting the scales, the human computer can "prove" nearly anything.

Those developing computer-based trading systems face the same problem.  The modern software makes it easy to include many variables — too many!

Some Helpful Illustrations

Bill Rempel missed the Kentucky Derby by a few days, but his story highlighting horse race handicappers is excellent.  A group of handicappers were tested, using gradually increasing amounts of information.  The extra data increased their confidence, but not their performance!  (Read the entire discussion.)

Bill discusses Occam’s Razor and points out the importance of reducing the number of independent variables:

I use this paring down or pruning technique at work as well as when
examining trading strategies or opportunities. My first question, when
faced with complex models, has for a long time been “I wonder how many
of those variables actually do most of the work?”

This is pretty convincing to us, since Bill sounds just like our own Vince Castelli.  It is easy to develop a model using all of the available data and lots of variables.  You will generate a perfect "post-diction" but not anything useful for prediction.

The result:  Over-fitting and over-confidence, a dangerous brew!

Unfortunately, consumers of system strategies, including a few big-time "gatekeepers" we have met, have become accustomed to seeing eye-popping (and unrealistic) results.  They apply an automatic discount, regardless of the methodology employed.

The TCA Model Applied to the S&P 500

For the purposes of comparison, the chart below shows our TCA Model (Trend, Cycle, Anticipation) as applied to the S&P 500.  Without giving away the store, we can say that the model uses a relatively small number of variables — some designed to choose between trend and cycle, and others representing indicators for each.  Much of the power comes from advanced techniques for filtering and smoothing data, thereby improving signal to noise.  The chart below is not a back-test, but the signals actually used in trading during the last year.

The overall performance shows a gain of about 6% during a time when the S&P declined by a few percent.  It accomplishes this while reducing risk by staying out of the market for significant periods.

A key point is that the model gets the investor into the market to enjoy the big moves.  The cost?  There are some losses at times of rapid changes or churning.

Finding the big moves is very important.  Some traders have trouble joining in when the market has already made a move.  They are reluctant to "chase."  It is difficult to show gains when missing the big rallies.

Anyone interested in trading systems should join us as regular readers of The Rempel Report, where he updates and reports on several interesting trading systems.  One of these is similar to our own sector rotation approach.

TCA-ETF Update

Each Thursday (a day late this week) we share with the investment community a recent report from our ETF ratings.  We have been doing this in real time for eight months.  Our purpose is partly to gain visibility for the approach (free report available on request), but also as information for other ETF traders, and most importantly to provide a laboratory for others trying to develop trading systems.  We discuss the issues surrounding system development in many of the articles in this series.

As we noted last week, we have expanded the ETF universe, and we seek more additions.  Adding more targets is helpful, as long as they can be shown to have characteristics suitable for one’s model.

The current ratings show some dramatic changes from recent weeks, and include one of the new ETF’s, KOL.