It's not the numbers, it's how you use them

I'm a firm believer in using numbers to understand what's happening in operations in all sorts of fields, and that careful, thoughtful analysis can provide good information about what works and what doesn't. But it's also easy to use numbers and come up with half-baked, or wrong, or silly ideas. A major culprit is the US News and World Report rankings of colleges and universities, which slam together a few metrics to come up with universal marks. Malcolm Gladwell begins to take apart the process in the current issue of the New Yorker; the article is available, behind a paywall, here.

Sometimes, you need to understand the context. Timothy Noah, generally a thoughtful and careful journalist, has done some digging around a report about an increase in aircraft incidents involving air traffic controllers. The article and associated comments are here. Commenters have pointed out that Noah neglected to mention the denominator (the huge number of controlled flights in the US, making the changes Noah is discussing statistically insignificant), and that long-run time series data are missing (two years do not make a trend). I'd also add that joining two databases often requires reclassifying the data stored in one, or both, and I read Noah's article as implicitly suggesting the FAA do so. But that might lead to more inconsistency, not less, as different people will view, and code, incidents differently.