The Dunning-Kruger Effect: Can We Profit?
During my mini-vacation i got some interesting email from two intellectual friends, both related to the Dunning-Kruger effect. Neither friend seemed to be aware that this has been a basic theme of my blog for nearly five years, but both knew that I would be interested.
Even when on vacation, I always read Abnormal Returns. Citing last weekend's New York Times article on the subject, Tadas seized the theme to review analyst reaction to the impact of the oil spill on BP earnings.
This was a nice post, highlighting the dilemma and possible bias of analysts. Let us see what we can add.
Here is an additional slant from the provocative blog, You Are Not So Smart (now added to our featured list):
Have you ever wondered why people with advanced degrees in climate science or biology don’t get online and debate global warming or evolution? Yet, people without a degree in psychology will write 1,200 words about a psychological bias. The less you know about a subject, the less you believe there is to know in total. Only once you have some experience do you start to recognize the breadth and depth you have yet to plunder.
I wonder about this question all of the time, and you should, too. Here are some simple propositions that I have advanced for years. I feel like it is a losing battle.
- Every source of information is good for something. The problem is what it is. I have given examples as simple as talking to cab drivers. (Obviously, I need one of our new resource pages on this).
- Most sources pontificate widely and loudly, reaching far beyond their area of expertise. Baseball fans should revisit my article on advice from "the Splendid Splinter" about waiting for the right pitch. Stay in what Ted called the happy zone.
- A valuable investor skill is identifying the real experts. Hardly anyone can do this. Evaluating expertise is difficult.
In my featured blogs, for example, I make personal decisions about whether the source is strong on a given topic.
A Few Examples
Every day you can read or watch a parade of pseudo-experts. Here are things they will say:
- Models are without value, "mark to myth", or error prone. This comes from people who have never built a model, did not take any class, and claim not to do forecasting. They do not understand that the process of building a formal model involves evaluating accuracy.
- Disparaging economic experts. This is a favorite of bloggers who talk pop economics. They are like the psychologist example cited above — they are confident that knowledge they do not have is irrelevant to the problem. How do they know?
- Giving lessons on public policy. The investment punditry enjoys pointing out how dumb those in Congress seem when on TV. They are blissfully unaware of how stupid they themselves seem to anyone who understands government. The punditry is used to calling the shots and making split-second decisions using a "rational" model. These pseudo-experts do not understand compromise, coalition-building, or political accountability. Their education started and ended with Mr. Smith Goes to Washington. When a market guy takes this approach, and lectures to government officials, he is really not taken seriously.
Most of the Internet audience is from a certain demographic and ideological audience that applauds sources and articles like my examples. They all cater to pre-conceived notions about knowledge, expertise, and decision-making. As a result, the articles are popular. This is good for TV ratings and page views.
Briefly put, most pundits and their readers are classic examples of the Dunning-Kruger effect. They are all blundering and they do not know what they do not know.
Anyone who is really serious about understanding the economy, public policy, and investing needs a special skill:
What can I really learn from this source?
I know that many readers are suspicious of information from popular sources like Jim Cramer, Larry Kudlow, Barney Frank or even Ben Bernanke. This is a mistake. Each is an expert on something important. I would be interested on reader comment on this. Please take the question seriously. What is the real "happy zone" for each source?
Tomorrow's Application: The Fed
There are a number of excellent sources on the Fed, policy options, and the likely choices. These sources share a number of traits including (but not limited to) the following:
- They have economic knowledge, supported by data. This should include some formal training, since that is how knowledge is acquired. If you do not understand this, you are a permanent resident of the Dunning-Kruger group.
- They have actual responsibility and decision-making experience. They have worked in policy-making roles, sometimes as key consultants to government agencies.
- They have no specific political agenda. They are invited to comment based upon expertise, not to "represent a viewpoint."
Here is another reader challenge: From the Fed commentators, see how many sources you can find who meet the criteria listed above.
To do so, you will have to fight your way through those who lack any real credentials, but nonetheless hold strong opinions about the stupidity of actual decision-makers.
Investment Conclusion
The Fed needs to stimulate lending, increasing the velocity of money. Meanwhile, most of the criticism from the punditry points to the monetary base and emphasizes inflation fears.
I expect no policy change and little change in language. With no sign of inflation, the Fed will continue to emphasize the greater concern — deflation. This is more difficult to correct if a mistake is made. Briefly put, the Fed is going to support the economy until we have a clearer sign of self-sustaining growth.
This is the thing that many miss in their doom-and-gloom forecasts, just as they did in 2008. Policy makers will not sit idly by. No one predicted the wide range of Fed responses and loan facilities. Essentially, these pundits got it completely wrong when it came to predicting public policy outcomes.
When I invest for the long term, I find it helpful to have the government and the Fed on my side.
[full disclosure — I remain officially neutral on the Ticker Sense poll and slightly short in trading accounts]
Prof,
I appreciate your point of view, and I am grateful that you blog. I have changed my perspective as a direct result of some of your posts.
Still, I maintain one large bit of skepticism toward your work. My gut-reaction is to doubt the experts, but you often suggest we should only listen to the experts. But you also suggest listen to the likes of taxi-drivers. I wish you would expand on this topic, as I am not following you logig (my weakness).
Thanks.
At least in your case, if you say you talked with a taxi driver we can have confidence that you really did so (unlike some famous pundits we could name).
I do like to listen to experts, and the point is well taken.
However, I find it hard to take economists seriously. Many of the models they build are so flawed its comical. Often they use the wrong math to try and define financial markets (Gaussian math doesn’t cut it). They should try reading Mandelbrot. Secondly, they like models that create some ethereal equilibrium: so far from the real world. Samuelson’s direction for modern economics has been an unmitigated disaster.
I would rather go with Keynes and make an effort to understand that we don’t understand much about the future.
Russ — It is fine to be skeptical of experts and to use critical thinking. This is quite different from dismissing an entire body of knowledge as irrelevant.
Interpreting information starts with understanding expertise. A taxi driver knows a lot about what is going on in his community. A CEO knows a lot about his own company. Neither one of them is an expert about economic prospects. That would not stop most in financial media from sticking a mike in front of them and asking about the chances for a “double dip.”
Many bloggers are excellent in their specialized area, but then wander into giving their opinions about everything.
If you look around, you will see plenty of examples. It is not that experts are always correct in their predictions, but that is usually a good place to start.
Thanks for the good question.
Jeff
John — We are both experienced with the options world where it is possible to adjust pricing models to fit the actual distributions. In fact, that was one of my jobs for CBOE market makers in the OEX.
I am working on an article on this topic, so I am especially interested in your comment.
If you change the environment to an economic forecast, the problem is quite different. Knowing that you have a “fat tail” for example, will not necessarily change your forecast. It changes the error band, however you define that. There may be a significant chance of a large “event”. but that also does not fit a forecast.
So — if you and I were to improve on standard economic forecasting, what would we suggest?
Most importantly, you know enough economics to engage in constructive critical thinking. Most of the ‘pop economist’ crowd dismisses the real economists because it reduces discussion to slogans and anecdotes — their weapons of choice 🙂
Thanks again!
Jeff
2 questions related to themes in this post:
1. http://welling.weedenco.com/%28X%281%29S%28aml53h45324fwrrp5qefto2r%29%29/default.aspx?menugroup=reprint&AspxAutoDetectCookieSupport=1
“Well, the U.S. government hasn’t invited me to do anything directly – yet. However, I go to Washington once a year because I used to have a very generous doctoral fellowship from the Board of Governors of the Fed in the early ’80s, and it is my way of paying back that debt. Every year, I give a presentation to the Fed. I must say that for a long time, ever since I started talking about this concept of a balance sheet recession, I was bashed and bashed and bashed, every time I’d give a seminar at the Fed. Only in the last three years or so have they begun saying, maybe you are right that this kind of thing can actually happen. But many Fed staffers now are aware of my argument and what has to be done to deal with balance sheet recessions. Meanwhile, the CSIS, the Center for Strategic International Studies, which is really more of a national security, rather than an economic, think tank, has also invited me to present at two big events it has sponsored in Washington. The most recent, last year, exposed a lot of Congressional staffers to my ideas. So, at least some people in the capital are aware of what I have been saying. What’s more, I’ve recently seen a major change in tone from Larry Summers, the director of the White House National Economic Council.
How does one identify the real expert here? Is it Richard Koo? Or is it the Fed staffers and Larry Summers who consistently rejected his analysis and recommendations? In other words, when we are dealing with a group all of whom have the right credentials yet have viewpoints that are 180 degrees apart, how does one assess the expertise?
2. http://www.ritholtz.com/blog/2010/06/ecri-weekly-leadinbg-indicators-widely-misunderstood/
“We aren’t entirely surprised. Virtually the entire analytical community has been trained to believe that any and all quantitative approaches to forecasting must involve models, i.e., simplified representations of reality. The idea that there could be rigorous quantitative approaches that are not model-based seems to be entirely beyond their ken. Their understanding of analytical techniques is so model-soaked that it reminds us of an insightful comment from psychologist Daryl Bem: “It takes a very intelligent and non-parochial fish to realize that his environment is wet.”
How should I intepret these comments? Is Mr. Achuthan a qualified expert?
Mike C — Concerning the first case, they are all experts. Koo might be most expert on a specific topic, and he is gaining attention. The relevance of his viewpoint may be especially important in the next year or two. Many support the indicated policy.
Even if this were not true, I never stated that identifying experts would answer all of our questions. In the avalanche of information, it helps to have a filter. I recommend filtering out people who tell you that economics (or forecasting, or behavioral psychology, or political science, or research methods, or modeling) is irrelevant to the discussion when they themselves lack the relevant background. That is precisely what the theory points out.
I’ll circle back to point 2 after I consult my copy of the book (which is at home) and can review the BofA report.
Thanks for the links.
Jeff
While on the one hand, dismissing commentators or refusing to budge from your position as it doesn’t support your own bias is a well studied case in psychology. I think what you’re missing is the public dissemination of information by “experts” which are not held to any real accountability, is suspect. In many if not most there is often an agenda. A money manager talking his book, or steering the masses ala Soros and his gold comment. “Experts” cant escape YouTube..Barney Frank and his “I’m willing to roll the dice on housing” speech in 2004. Maxine Waters speech in 2005 “If they ain’t broke, why you trying to fix em'” comment about Fannie and Freddie. Or how about Kudlow’s “goldilocks” economy pitch along with other ala 2006-2007. Or the best, Bernanke’s classic Youtube clip of “Subprime and housing defaults will not exceed 40Billion and will not spill over into other areas of the credit market”….Maybe I misunderstand your post, but as Maxine says “I ain’t buyin it”…
Great post as always, thanks for provoking some thought. For me, it boils down to whether I have room to consider the opinion of an “expert”. I don’t doubt credentials, but I often doubt the relevance of any one opinion as well as my ability to absorb the information into a profitable plan. With an infinite supply of sources, I tend to focus my consumption on process-oriented advice and let market action steer me in planning outcomes. You do a great job of respecting both, hence your work ranks higher on my consumption scale.
In all honesty, it’s just too difficult to build a model describing our economy and it’s increasing complexity. I would suggest limited economic forecasting, and only those things that can be modelled.
Most economic modelling just follows the herd. Is there one Wall Street analyst predicting a down market this year? I know it’s bad for business to suggest such a thing, but it’s naive to not at least assume it could happen (it’s 2006 and housing prices can’t fall can they???).
I sat in a talk by Joseph Stiglitz (not my favorite economist really, I like Shiller) who described the IMF as not modeling in the effect of monetary policy. I mean wtf?
Following up on Shiller, it’s extremely difficult to model in animal spirits. For example, how is it people are modelling in growth for Europe? They are creating a massively deflationary environment that could crimp growth for years (all becuase of the fear of Weimar, when they should fear a Japanese period instead).
I guess I would say: What do we really gain from crowded herd economic forecasting? I don’t see it.
Thanks Jeff. After seeing Donald R’s quote years ago, I called it “The Rumsfeld Paradox”. As an IT system analysis geek, I’ve been fighting this every day for the last 20 years.
My solution so far was “more information” but thanks to unlimited cable channels with time to fill, and democratic Internet access, there’s too much noise. Aggregating and crowdsourcing fail. Picking your own experts seems the way so far (I really like your criteria #2 and #3). Is it best to pick 2-3 sources and ignore the rest?
Just yesterday I had an interesting debate in my history of psychological thought class that runs almost along similar lines. A classmate of mine forwarded his argument that scientists never do work that cannot be applied. I quickly shared my disagreement because research is not always about maintaining what will benefit the status quo but also to pave the way for finding knowledge that pushes the boundaries of common thought. One cannot deem a particular work as useless or useful not unless that person has sufficient knowledge and mastery to proclaim it so.