All Hail Numeracy

John Allen Paulos, who writes about numbers as well as anyone around, has just published another excellent piece in the Times Magazine, this one about the strengths and limitations of relying on data to make decisions in the modern world. The underlying message: provenance matters, more than just about anything. And, furthermore: “No method of measuring a societal phenomenon satisfying certain minimal conditions exists that can’t be second-guessed, deconstructed, cheated, rejected or replaced.”
As he did here, Paulos provides an excellent example of tricky statistics from the medical realm:

Medical researchers face similar problems when it comes to measuring effectiveness. Consider the temptation to use the five-year survival rate as the primary measure of a treatment for a particular disease. This seems quite reasonable, and yet it’s possible for the five-year survival rate for a disease in one region to be 100 percent and in a second region to be 0 percent, even if the latter region has an equally effective and cheaper approach.
This is an extreme and hypothetical situation, but it has real-world analogues. Suppose that whenever people contract the disease, they always get it in their mid-60s and live to the age of 75. In the first region, an early screening program detects such people in their 60s. Because these people live to age 75, the five-year survival rate is 100 percent. People in the second region are not screened and thus do not receive their diagnoses until symptoms develop in their early 70s, but they, too, die at 75, so their five-year survival rate is 0 percent. The laissez-faire approach thus yields the same results as the universal screening program, yet if five-year survival were the criterion for effectiveness, universal screening would be deemed the best practice.

What are the weapons best deployed to fight data smog?
Footnotes, I guess, and transparency, and a generally higher level of numeracy among the populace. That’s been Paulos’s mission for some time now. He is of course the kind of service provider who could put himself out of business if he does his job too well. But given the current level of, say, simple financial literacy, I doubt that is much of a threat.