A certain aspect of Mr. Kristof’s New York Times article dated 3/26/09 , “Learning How To Think” really caught my attention.
In the article he provides results from research conducted by Philip Tetlock, a professor at the University of California, Berkeley. The research is conducted on Expert Opinion provided by Experts and its accuracy. Dr. Tetlock tracked 82,000 predictions by 284 experts over two decades. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.
The result? It appears the experts were, on average, only a tiny bit better than random guesses! More confounding, it made no difference what their education level or experience level was.
Next, to quote from the article:
“Indeed, the only consistent predictor was fame — and it was an inverse relationship. The more famous experts did worse than unknown ones. That had to do with a fault in the media. Talent bookers for television shows and reporters tended to call up experts who provided strong, coherent points of view, who saw things in blacks and whites. People who shouted — like, yes, Jim Cramer! “
I consider it important to interject a note here: Mr. Kristof and Dr. Tetlock seem to be soley focused on opinionators in the media and the opinions they aired in public. However, the way the column is written it is misleading since it seems to undermine all expert opinion. That is not the point I agree with or I am focused on.
I am more interested in creating more foxes vs. hedgehogs, you will soon see what I mean.
Quoting the paragraphs that are most important for the purposes of this post:
“Mr. Tetlock called experts such as these the “hedgehogs,” after a famous distinction by the late Sir Isaiah Berlin (my favorite philosopher) between hedgehogs and foxes. Hedgehogs tend to have a focused worldview, an ideological leaning, strong convictions; foxes are more cautious, more centrist, more likely to adjust their views, more pragmatic, more prone to self-doubt, more inclined to see complexity and nuance. And it turns out that while foxes don’t give great sound-bites, they are far more likely to get things right.
This was the distinction that mattered most among the forecasters, not whether they had expertise. Over all, the foxes did significantly better, both in areas they knew well and in areas they didn’t. “
This I believe in.
Mr. Kristof ends his column (not quoted here) by talking about putting a system in place that measures the success ratio of experts that are paraded out in the media. While important, I think he missed the opportunity to make a more fundamental point: How do we train people to be more like foxes than hedgehogs?
Filed under: Education | Tagged: economy, Education, expert, kristof, media, NYT, society, tetlock | 3 Comments »