Businessweek Archives

The Problem with Prediction Registries


? Rising Wages in China |

Main

| My Take on the Coming Housing Bust ?

March 21, 2006

The Problem with Prediction Registries

Michael Mandel

David Brin (whom I like very much) has advocated 'prediction registries'-that is, keeping track of the predictions of experts and pundits, so we can see which ones were right and which ones are wrong.

A tempting idea. But Brin points out that such registries are not around the corner:

Before such an archive can be built, somebody must fund several man-years of R&D, all toward a system that can be seen as credible even when partisans see their own side proved wrong! That's not easy to do. It will take breakthroughs in the representation of opposing ideas. But the payoff could be stunning.

There's an even more basic problem with prediction registries--deciding on the time frame of the prediction. Consider, for example, the repeated prediction by most economists that the trade deficit is unsustainable, and will lead to a massive slide in the dollar. The December 4, 2004, cover story of The Economist was called "The Disappearing Dollar." The basic idea of the story was simple:

Over the next few years it seems an excellent bet that there will be a large drop in the dollar.

In many ways, this is the classic wrong call. Almost as soon as the The Economist was on the stands, the dollar started to appreciate against the euro and the yen (the dollar hit its low against the euro on 12/30/04, and has risen by 12% since then. The dollar hit its low against the yen on 1/14/05, and has risen by about 15% since then).

However!! The actual prediction was "over the next few years." So if the dollar dropped sharply in, say, 2008, the writers of the article would be entitled to say that they were right.

And if you think it's just journalists who suffer from this problem, note that the same article cited a 2004 paper by Noriel Roubini and Brad Setser (hi Brad!), which said:

The rapid deterioration of US net external debt position implied by large trade and current account deficits cannot continue indefinitely.

and

Even if the United States continues to be able to borrow on terms that other, comparable, debtors could not imagine, our analysis suggests that the U.S. is on an unsustainable and dangerous path.

and

the tensions created by this system are large, large enough to crack the system in the next three to four years.

Note that Roubini and Setser give themselves a window of the "next three or four years" for all those horrible things to happen.

Certainly nothing horrible has happened over the past 15 months or so, since they wrote the paper. The trade deficit has continued to grow, the dollar has appreciated, the U.S. economy has chugged along as happy as a pig in the mud. So in the short run the trade deficit was clearly sustainable. Not only that, it's unlikely that Roubini and Setser would have predicted an appreciation of the dollar.

However!! If there was to be a sharp downward depreciation in the dollar anytime between now and 2008, it would be consistent with the Roubini-Setser prediction. Moreover (and here's where the fun part comes in), even if there was a gradual depreciation of the dollar so that the trade deficit came down gently, I suspect that Roubini and Setser would deem that consistent with their prediction of "unsustainability" as well. Perhaps Brad could address that question.

In any case, I don't really mean to pick on them, though I do mean to pick on the Economist. It's just very hard to identify truly falsifiable predictions "in the wild" as it were.

P.S. As far as I know, nobody has ever systematically studied the accuracy of cover stories in either The Economist or BusinessWeek.

02:47 PM

Future

TrackBack URL for this entry:

http://blogs.businessweek.com/mt/mt-tb.cgi/

Listed below are links to weblogs that reference The Problem with Prediction Registries:

? The Problem with Prediction Registries from A L E A

In The Problem with Prediction Registries Michael Mandel says “As far as I know, nobody has ever systematically studied the accuracy of cover stories in either The Economist or BusinessWeek”.

The creator of the magazine cover indicator,... [Read More]

Tracked on March 22, 2006 07:21 AM

The Economist's "The World in 2006" has a nice two-page article by Niall Ferguson, called "Looking back on the future", on the various "hits and misses" on the publication throughout the years.

Posted by: sredni_vashtar at March 22, 2006 10:32 AM

Reminds me of the analysis of Delphic predictions and how successful they can be when they are not predictions but prescriptions.

Posted by: Lord at March 23, 2006 03:03 PM

Whichever political party is in the majority, by belief if you are not, you will be bearish on the economy, GDP, inflation, and interest rates.

I fully expect unexpected results to occur.

For instance, the 30 year rate may be the same or lower than the 15 year rate. I also expect interest rates to reach new relative lows compared to 2004 and 2005 with the Prime Rate 1% higher than fixed rate mortgages. I still expect Oil to be $60+/barrel for another year with no recession until 2008 or later.

The Middle East will soon become the least of our worries.

Posted by: Frank Drake at March 25, 2006 12:36 AM

A good project for the Wikipedia, the peer editing online encyclopedia folks to get behind. There may be! (fact) investment in economic visions. A peer editing program lets all sides cut to the facts.

There can be backward review of past predictions, that can kick start the economic data base. There are really ten sides to every economic view, so ten sets of editors may be needed.

Posted by: Mike Reardon at March 25, 2006 03:08 PM

Michael -- I took up your challenge.

Posted by: brad at March 28, 2006 11:11 AM


Later, Baby
LIMITED-TIME OFFER SUBSCRIBE NOW

(enter your email)
(enter up to 5 email addresses, separated by commas)

Max 250 characters

 
blog comments powered by Disqus