The Free Market Project

Markets, statistics and the reporting of both — explained plainly

Special Report

Ten Recurring Misreadings of Economic Statistics

A brass magnifying glass resting on a blank sheet of cream paper, focusing a disc of sunlight
A brass magnifying glass resting on a blank sheet of cream paper, focusing a disc of sunlight

Economic coverage makes the same small set of errors repeatedly. They are not confined to any country, outlet or political tendency, and most are not really errors of fact — they are errors of definition, produced by writing about a measurement without describing what it measures. Each one below is common enough that you will see it within a week of looking for it.

1. Treating an average as a typical case

Income, wealth and earnings distributions are strongly right-skewed, so the mean sits well above the median and moves in response to changes at the top that most people never experience. A story reporting that "average household wealth rose" may be describing an economy in which the median household got poorer. Whenever an average appears without a median beside it, the distribution is doing work the sentence is not acknowledging.

2. Comparing money across years without adjusting for inflation

Any comparison of monetary amounts across time that has not been inflation-adjusted is partly a statement about prices. "Record spending", "highest revenue ever" and "the largest budget in history" are close to automatic in any growing economy with any positive inflation, and by themselves carry almost no information. Real terms, or a share of national income, are the meaningful comparisons.

3. Confusing a rate of change with a level

A falling inflation rate means prices are still rising, more slowly. A slowing rate of job growth means employment is still increasing. A shrinking deficit still adds to the debt. Each of these is reported as a reversal at least once a cycle, and each is a deceleration rather than a decline.

4. Ignoring the margin of error

Survey-based statistics come with published confidence intervals, and for a single month's change the interval is often wider than the change itself. A movement that cannot be statistically distinguished from zero is regularly reported as a turning point. The interval is in the release, one page past the headline table.

5. Reporting first estimates as if they were final

Output, employment and trade figures are provisional on release and revised as more source data arrives. Revisions are frequently larger than the month-to-month changes being discussed, and they arrive weeks later with a fraction of the coverage. A reader following only first estimates holds a noisier and occasionally inverted picture of what happened.

6. Mistaking gross for net

In a large economy, millions of jobs are created and destroyed every month. A widely covered layoff announcement of even tens of thousands is small against those gross flows, and says little about the net change. The same confusion appears with firm closures, trade flows and investment: the visible event is a component, not the total.

7. Choosing the start date that makes the point

Volatile series can be made to show almost anything by selecting the base period. Measuring from a peak produces a decline; measuring from a trough produces a boom. This is rarely deliberate — the natural comparison points are memorable dates, and memorable dates are usually extremes. Multi-year charts are the defence; single percentage changes from an unstated base are the warning sign.

8. Treating a correlation in a headline as a cause

Two series moving together is the starting point of an investigation, not its conclusion. Economies are dense with common drivers, and confident causal claims in aggregate data are hard to establish and require careful identification. The strongest signal that a claim is doing more work than the evidence supports is a mechanism stated in one clause.

9. Reading an accounting identity as a verdict

Several widely quoted figures are definitional rather than evaluative. A trade deficit is matched by a capital inflow; a government deficit is matched by private and foreign surpluses; savings equal investment in the national accounts by construction. These identities describe how the books balance, not whether an outcome is good. Treating "deficit" as a synonym for "failure" imports a connotation the accounting does not carry.

10. Quoting a forecast as a fact

Projections about growth, costs and revenue are model outputs conditional on assumptions, and they are usually published with sensitivity ranges. In coverage the central estimate becomes a number and the range disappears. When a policy is said to "cost" a figure over ten years, that figure is nearly always a conditional projection from a stated baseline — both of which are in the source document and neither of which usually survives into the article.

Why these recur

Every item on this list makes a sentence shorter and more definite. A caveated version of the same claim is longer, hedged and less publishable — and in a piece with a strict word count, the caveat is the first thing cut. The mechanics behind that are set out in The Balance Sheet.

The remedies are ordinary. Read the release rather than the summary where you can: the Bureau of Labor Statistics publishes a Handbook of Methods explaining how the underlying statistics are collected, and FRED lets you plot almost any series over any period in about a minute, which defeats items 2, 3 and 7 on this list by itself. For the definitions behind the employment figures specifically, see how the jobs numbers are built.