Context: it's how you use the numbers, not what they are

The normally extremely clear writer James Fallows (his terrific blog on politics, technology, and beer, among other things, is here) has posted a letter from one of his readers. The full blog post is here; the excerpt I'm interested in is:

First, our public policy discussion has become too wonkish, by being entirely focused on measurable outcomes at the expense of all others.  (Another example: the health care debate, the vast majority of which was about costs instead of the moral imperative of universal health care)...

It may just be the writing (to repeat, it's not Fallows' but a reader's comment), but this strikes me as an example of someone blaming the numbers, as opposed to the interpretation of the numbers, for the politics. Sometimes a mathematical model is our best chance of understanding what is happening in the world (even if that understanding is weak). But it's our interpretation of the numbers - the context we give it - that guides how we use them. Numbers are never just numbers.

Obesity studies

You may have read yesterday about a study published in the Journal of the American Medical Association concluding that, although obesity overall was associated with higher mortality rates, overweight (identified by Body Mass Index, or BMI) was associated with statistically significant lower mortality rates. This conclusion has been trumpeted as vindication for the view that we are, perhaps, overfocused on the importance of weight loss for good health. See, for example, here, here, and here.

This conclusion depends, of course, on BMI being a good proxy for mortality risk. There's some evidence that it's not. One drawback, according to Well-Being Wire, is that BMI fails to account for an individual's exercise habits - you can be fit but fit with a high BMI, and slender but sedentary with a low BMI. In those cases, BMI won't tell you who is healthier.

In any case, the study is not an endorsement of heaviness or weight gain. The authors conclude: 

Relative to normal weight, obesity (all grades) and grades 2 and 3 obesity were both associated with significantly higher all-cause mortality. Grade 1 obesity was not associated with higher mortality, suggesting that the excess mortality in obesity may predominantly be due to elevated mortality at higher BMI levels.

And even the next sentence, "Overweight was associated with significantly lower all-cause mortality," needs to be read with some skepticism. As Lindsay Abrams writes at TheAtlantic's web site:

the study fails to take into account any of the various other measures used to assess health. It ignores blood pressure, blood sugar, and cholesterol -- high levels of all are directly associated with a variety of chronic conditions and diseases -- not to mention mental health and life satisfaction scores.  

She provides some interesting pictures that make the point about BMI, and concludes:

While in the most basic of ways, it makes sense to pay attention to the number on the scale, it only gives us one metric of health that, if not understood in context, is basically useless. If we could get used to looking at weight more holistically, in terms of overall health, the link between BMI and longevity wouldn't be so shocking.

So read the study. Think about it. And then continue to work out.

Adam Davidson on statistics and narrative

If you haven't yet read Adam Davidson's article "God Save the British Economy" in yesterday's New York Times Magazine click on the link and read it. It's a very clear explanation of the British government's economic austerity policy and related criticism. From this blog's perspective it's also an explicit and important description of the use of statistics in forming narrative - the narrative that we use to make sense of the world.

Here's one example from the article:

Economics often appears to be an exercise in number-crunching, but it actually resembles storytelling more than mathematics. Before the members of the Monetary Policy Committee gather for their monthly meeting, they sit through a presentation from the Bank of England’s economic staff. The staff members take the most recent economic data — G.D.P. growth, the unemployment rate and more subtle details gathered from interviews with businesspeople throughout the country — and try to fashion it into a narrative. Does a sudden spike in new factory orders represent a fundamental shift, or is it just a preholiday blip? Do anecdotal reports of rising food prices herald a period of inflation, or is it the result of a cold snap? Which story feels truer?

I've often talked about the importance of thinking about the statistics we use, understanding the context and thinking about the cognitive choices we are making when we develop and interpret the statistics. In this story Davidson dissects a set of choices. Read it and think about it - and tell me what you think in the comments.