A reader named Dave makes an important point in a comment on my item last week about the supposed 4.5% rise in housing starts in December.
Dave says—accurately—that we don’t really know whether housing starts rose at all, because the margin of error in the Census Bureau’s survey was so large. He thinks we should have admitted our ignorance. In Dave’s words: “Such statistically negligent reporting continues to amaze me!”
He's right. I didn't mention the margin of error at all in my blog item, instead treating the "increase" like a settled fact. In my defense, I was hardly alone: just about every journalist and financial analyst and economist who wrote about the Census Bureau report stated that housing starts rose and made no mention of the margin of error.
If we had been scrupulous, we would have written something like this:
Census Bureau Doesn't Know if Housing Starts Rose or Fell in December
Confusion over the state of the housing industry continued today as the Census Bureau once again said that it doesn't know whether the rate of construction of homes rose or fell in the most recent month. The bureau estimated that starts rose 4.5%, but the margin of error was plus or minus 8.8 percentage points. The same thing happened for October, where the estimated increase was 6.7% but the margin of error was plus or minus 10.1 percentage points. On the bright side, we're pretty sure that starts really did increase in September, because the estimated increase then (14.6%) was bigger than the margin of error (plus or minus 7.6 percentage points).
As this exercise goes to show, the strictly scrupulous approach can be tedious. On the other hand, it has the important advantage of sticking to what's actually known. So I agree with Dave. I'm going to make more of an effort in the future to point out when numbers are particularly squishy because of sampling error.
For those who like reading the fine print, here is what the Census Bureau stated in a footnote on the first page of its housing starts news release:
In interpreting changes in the statistics in this release, note that month-to-month changes in seasonally adjusted statistics often show movements which may be irregular. It may take 4 months to establish an underlying trend for building permit authorizations, 5 months for total starts, and 6 months for total completions. The statistics in this release are estimated from sample surveys and are subject to sampling variability as well as nonsampling error including bias and variance from response, nonreporting, and undercoverage. Estimated relative standard errors of the most recent data are shown in the tables. Whenever a statement such as "2.5 percent (plus or minus 3.2%) above" appears in the text, this indicates the range (-0.7 to +5.7 percent) in which the actual percent change is likely to have occurred. All ranges given for percentage changes are 90-percent confidence intervals and account for only sampling variability. If a range does not contain zero, the change is statistically significant. If it does contain zero, the change is not statistically significant; that is, it is uncertain whether there was an increase or a decrease. The same policies apply to the confidence intervals for percentage changes shown in the tables. On average, the preliminary seasonally adjusted estimates of total building permits, housing starts and housing completions are revised about one percent. Explanations of confidence intervals and sampling variability can be found on our web site listed above. [http://www.census.gov/newresconst]
By the way, Census uses 90% confidence intervals. If the bureau used a 95% confidence interval, as many social scientists do, the margins of error would be even greater.
Dave--thanks for the comment.
BusinessWeek editors Chris Palmeri, Prashant Gopal and Peter Coy chronicle the highs and lows of the housing and mortgage markets on their Hot Property blog. In print and online, the Hot Property team first wrote about the potential downside of lenders pushing riskier, "option ARM" mortgages and the rise in mortgage fraud back in 2005—well ahead of many other media outlets. In 2008, Hot Property bloggers finished #1 in a ranking of the world's top 100 "most powerful property people" by the British real estate website Global edge. Hot Property was named among the 25 most influential real estate blogs of 2007 by Inman News.