More about my critique of macro forecasting

Posted by: Michael Mandel on June 28

Craig Newmark notes my critique of macroeconomics, and then writes “I wish he would document the claims.”

Well, here is the documentation for two of the critiques that I mentioned (all that I have time for right now, as I’m trying to get a cover story out the door). If anyone is interested in the others, I will be glad to provide them.

Claim #1) Year ahead forecasts of short-run growth, by and large, are terrible, because macroeconomists have developed no good way of forecasting structural productivity growth.

Response: Since 1995, the year-ahead forecast error in GDP growth has averaged 1.4 percentage points (details available on request). Since growth in that period averaged 3.4% per year, the average error was about 40% of the trend. That seems pretty poor to me.

Claim #4) The estimated “natural rate of unemployment” seems to move around unpredictably.

Response: According to a 2001 paper by Douglas Staiger, James Stock, and Mark Watson, “once one accounts for the univariate trends in the unemployment rate and in productivity, the 1990s present no price or wage puzzles. Thus, the task is to explain trend movements in productivity and in unemployment.”

Translation: If we could predict productivity growth, we could predict the natural rate of unemployment. Unfortunately, macroeconomists are terrible at predicting shifts in structural productivity growth.


TrackBack URL for this entry: http://blogs.businessweek.com/mt/mt-tb.cgi/

Reader Comments

James

June 28, 2005 06:20 PM

According to Samuelson, the CBO puts the natural rate of unemployment at 5.2%, despite the fact that the current rate is 5.1%, and nobody that I am aware of considers this a tight job market. While their report appears to be a well documented review of what has happened, it seems to be basically guessing as to how changes in the job market impact the present and future.

http://www.cbo.gov/showdoc.cfm?index=3367&sequence=0&from=1

Dave Tufte

July 1, 2005 01:09 PM

(This comment also appears at Newmark's Door).

Both you and Mandel have some good points and some bad ones.

The good point is that the macroeconomy is highly unpredictable. Everyone should be more cognizant of that fact, and we'd probably get better voting if people recognized that most of the macroeconomic performance that politicians lay claim to (or avoid) is just luck.

Going downhill from there is the implicit argument that the macroeconomy should be predictable. Certainly that's desirable, but is it a requirement for labelling macroeconomics as a respectable field? What if it is just inherently difficult? Using approximately the same sample I got quarterly mean real GDP growth to be 3.5%, with a standard error of 2.1%. That is a lot of variation - implying near certainty (plus or minus 3 standard errors) that quarterly real GDP growth will fall between -3% and +10%. That is a huge range when you consider the fact that a difference between years of 1-2% is easily noticable to the layman. A simple but reasonable forecast of this (made from a second order autoregression) can only reduce that standard error to 1.9%, and that range to about -2% to +9%. I'm not sure where Mandel got the 1.4% error - even though it does seem reasonable from my experience with more serious models - but it suggests that the range of certainty can be reduced to -1% to 8%. The Fildes and Stekler number you cite can't be directly compared to this, but it suggests a somewhat tighter range. My point here is that these forecasts are close to useless because the underlying series - which Mandel chose for the basis of his argument - is inherently unpredictable. If he had chosen something more predictable - like say the arguably more important level of real GDP as opposed to its growth rate - we would have a different story. Yet, after all this, those lousy forecasts are capable of reducing those ranges by the amount that the typical person can feel in their bones, so they are probably worthwhile.

The bad point is that it isn't appropriate for you guys to compare in percentage terms the 1.4% figure (one that from it's calculation is guaranteed to be positive) with the 3.4% figure that is an average of observations that are both positive and negative. It's OK to use that figure form some sort of confidence interval, but to claim that 40% is a meaningful number is not correct. In your defense, regular statistics books are good at making this point, but economics and business statistics tend to gloss over it (probably because this "coefficient of variation" calculation is misused in precisely this way in the field of finance).

Thank you for your interest. This blog is no longer active.

 

About

Michael Mandel, BW's award-winning chief economist, provides his unique perspective on the hot economic issues of the day. From globalization to the future of work to the ups and downs of the financial markets, Mandel-named 2006 economic journalist of the year by the World Leadership Forum-offers cutting edge analysis and commentary.

BW Mall - Sponsored Links

Buy a link now!