One of my very first posts was about Hans Rosling's "200 Countries in 200 Years" video, which shows the growth in income around the world across two centuries. Now here's a link to Rosling's own site, Gapminder.com. It contains links to many of his famous TED conference videos, including the "Magic Washing Machine," embedded below.
But Gapminder has a lot more than that. One of the best pages is the "Health and Wealth of Nations" chart that lets the viewer compare all sorts of metrics around the world. Interested in seeing carbon emissions per person by country? It's there, and also available by region and world wide. Health, education, population, energy, environment work - it's all there, and it's all accessible. There's even a guide for teachers. It's creative and useful. And fascinating.
Monday
More anecdotal evidence of global warning
As if any were needed: the New York Times sports section has an article today describing high altitude glaciers melting. If it weren't such an inappropriate adjective I'd call it chilling. But I suppose scary is the operative adjective. Glacierworks has a graphic photo illustrating the difference in one glacier. Here's how it looks now:
and here's how it used to look:
Source: Glacierworks
If you haven't seen one of the "Rivers of Ice" exhibits (underway now in London and Beijing) there's another US one slated at the MIT Museum in Cambridge, MA, opening in April.
and here's how it used to look:
If you haven't seen one of the "Rivers of Ice" exhibits (underway now in London and Beijing) there's another US one slated at the MIT Museum in Cambridge, MA, opening in April.
Thursday
Football and grades
You may have seen the story, since it's being widely reported, about a new study by three University of Oregon economists finding a decline in grades, especially among male students, when the school's football team is doing well. You can get the full study ($5) here.
Two things about the article: first, according to the NY Times, the study, while large, looked only at Oregon. Second, the article reports that the study found a correlation, though the comment in the article by the University of Oregon spokesman refers to causation. They're not the same thing. But the study points to yet another manifestation of the negative effects of big-time college sports. If you haven't read it already, read Taylor Branch's careful and thorough article in The Atlantic, "The Shame of College Sports."
Two things about the article: first, according to the NY Times, the study, while large, looked only at Oregon. Second, the article reports that the study found a correlation, though the comment in the article by the University of Oregon spokesman refers to causation. They're not the same thing. But the study points to yet another manifestation of the negative effects of big-time college sports. If you haven't read it already, read Taylor Branch's careful and thorough article in The Atlantic, "The Shame of College Sports."
Tuesday
Perspective matters - interesting chart
McKinsey's larger point, that a map can "correct the misimpression that a viewer's vantage point doesn't influence the way things look" is a valid one.
Monday
10 Years of Student Assessment
Michael Winerip's column in today's NY Times describes a series of statements, retractions, and, well, mistakes in New York State and City's attempts to assess public education. I've written about the City's efforts before, for example here (about the City's school rating system) and here (about teacher ratings). All I want to add now is that coming up with metrics is an iterative process, with lots of starts and do-overs. And, as Regents Chancellor Merryl Tisch continues to point out, the numbers have to mean something. I think that assessment of public education is something that can be done. I'm not sure that relying entirely on test scores is the way to do it.
One way to start fixing primary and secondary education is to provide good preschool education. If you haven't ever done so, take a look at the Highscope/Perry Preschool research material. It's pretty interesting, and, combined with the recent article about the declining number of "universal" pre-K spots (not to mention the inadequate numbers of child care spots) pretty compelling. If even some of the money spent on testing and test prep in the last decade had been spent instead on expanding high quality preschool education, New York's test scores might be showing a different trend.
I will be taking a little time off over the coming weeks for the holidays and don't expect to be posting as often. See you in January!
One way to start fixing primary and secondary education is to provide good preschool education. If you haven't ever done so, take a look at the Highscope/Perry Preschool research material. It's pretty interesting, and, combined with the recent article about the declining number of "universal" pre-K spots (not to mention the inadequate numbers of child care spots) pretty compelling. If even some of the money spent on testing and test prep in the last decade had been spent instead on expanding high quality preschool education, New York's test scores might be showing a different trend.
I will be taking a little time off over the coming weeks for the holidays and don't expect to be posting as often. See you in January!
Friday
"All models are wrong but some are useful"
UPDATE, Feb 9: Kaiser Fung's blog has a useful illustration of the variability point discussed below in a post about representations of LA's annual rainfall, here.
Kaiser Fung, a statistician and blogger (also here - I've written about his excellent site, "Junk Charts," earlier) has written a terrific book about what he calls "the statistical way of thinking." In it, he explains how to think about what happens when numbers don't lie. Here's a screenshot of the cover, from Amazon.
In five easily digested chapters, using events from newspaper reports or even everyday life to illustrate, he makes five basic points about statistical analysis that everyone should know.
1. The average isn't as important as the variability around the average - how large the variation is, how often it happens, and why are all more useful information. Fung illustrates this principle by discussing the wait times for rides at Disney theme parks. Think you have a long wait ahead of you? The "Fast Pass" allows you to come back at a set time, when numbers are lower. You feel as if you are not waiting, but of course you've gone and done something else while you waited.
2. Often, variability doesn't have to be explained by causes - correlation can be enough to make the information useful. Fung contrasts an epidemiological search for an outbreak of food-borne disease with consumer credit ratings. In the latter, correlation is a good enough reason to extend or deny credit. Epidemiologists need to be more cautious, and Fung reiterates Bradford Hill's nine aspects of cause and effect.
3. Not everyone or everything can be aggregated, and aggregation can mask differences. Fung points to differences in SAT scores among black and white students, which are masked when all scores are analyzed together. "Black students of high ability score the same as whites; the scores for low ability students of both races are also the same. And yet the average white student score . . . is higher than the average black student score. Due to superior educational resources, 60% of white students have high ability compared to only 20% of black students." The averages mask the weights, or the greater number of white students in the high-achieving group and of black students in the low-achieving group.
4. Making judgments based on statistics can force you to balance two types of error, over-inclusion and under-inclusion. Unfortunately, these two types are inversely related, so by lowering the chances of one you increase the chances of the other. Fung's example in this case is drug testing for athletes, and he shows that by setting the cutoffs for positive tests very high, there a lot of false negatives among tested athletes. Remember that next time you read a statement saying an athlete has never failed a drug test.
5. Statistical testing can help us decide whether available evidence explains an event. When sellers of Canadian lottery tickets turned up as winners at a rate considerably greater than chance would have us expect, Fung explains, it was unlikelihood of it happening (one in a VERY large number) that set authorities down a path to correct the wrong and change processes so that it was harder for vendors to cheat.
This is a clearly written, lucid book. After carefully setting out and illustrating the five main concepts, Fung demonstrates how statistical tools are often used together, going back to his original illustrations with a deeper explanation. It's very satisfying, and very clear. I highly recommend it.
Kaiser Fung, a statistician and blogger (also here - I've written about his excellent site, "Junk Charts," earlier) has written a terrific book about what he calls "the statistical way of thinking." In it, he explains how to think about what happens when numbers don't lie. Here's a screenshot of the cover, from Amazon.
In five easily digested chapters, using events from newspaper reports or even everyday life to illustrate, he makes five basic points about statistical analysis that everyone should know.
1. The average isn't as important as the variability around the average - how large the variation is, how often it happens, and why are all more useful information. Fung illustrates this principle by discussing the wait times for rides at Disney theme parks. Think you have a long wait ahead of you? The "Fast Pass" allows you to come back at a set time, when numbers are lower. You feel as if you are not waiting, but of course you've gone and done something else while you waited.
2. Often, variability doesn't have to be explained by causes - correlation can be enough to make the information useful. Fung contrasts an epidemiological search for an outbreak of food-borne disease with consumer credit ratings. In the latter, correlation is a good enough reason to extend or deny credit. Epidemiologists need to be more cautious, and Fung reiterates Bradford Hill's nine aspects of cause and effect.
3. Not everyone or everything can be aggregated, and aggregation can mask differences. Fung points to differences in SAT scores among black and white students, which are masked when all scores are analyzed together. "Black students of high ability score the same as whites; the scores for low ability students of both races are also the same. And yet the average white student score . . . is higher than the average black student score. Due to superior educational resources, 60% of white students have high ability compared to only 20% of black students." The averages mask the weights, or the greater number of white students in the high-achieving group and of black students in the low-achieving group.
4. Making judgments based on statistics can force you to balance two types of error, over-inclusion and under-inclusion. Unfortunately, these two types are inversely related, so by lowering the chances of one you increase the chances of the other. Fung's example in this case is drug testing for athletes, and he shows that by setting the cutoffs for positive tests very high, there a lot of false negatives among tested athletes. Remember that next time you read a statement saying an athlete has never failed a drug test.
5. Statistical testing can help us decide whether available evidence explains an event. When sellers of Canadian lottery tickets turned up as winners at a rate considerably greater than chance would have us expect, Fung explains, it was unlikelihood of it happening (one in a VERY large number) that set authorities down a path to correct the wrong and change processes so that it was harder for vendors to cheat.
This is a clearly written, lucid book. After carefully setting out and illustrating the five main concepts, Fung demonstrates how statistical tools are often used together, going back to his original illustrations with a deeper explanation. It's very satisfying, and very clear. I highly recommend it.
Tuesday
The pitfalls of evaluation
Who knew? The most e-mailed article on the Stanford Social Innovation Review website is dated 2006, and titled "Drowning in Data." Whether you're a funder or a service provider it's still a useful article to read five years after publication. Data collection, analysis, and reporting is hard, and the article outlines several of the reasons. One is that terminology is not yet standardized - my outcomes may look like implementation to you. Another is that organizations are often over-ambitious, wanting to know about outcomes that can occur only several years down the road without providing funding to develop that kind of information.
The main focus of the article is the disparate data requests generated by funders - and often, each requires its own form for reports even if data are similar. I've had to gather and report this kind of data, and it can be a problem, not least because the process may not generate data that's useful for managing a program.
The article distinguished what it calls "summative" evaluations: did the intervention "work"? from formative evaluations: does the evaluation help the organization "improve"? You can see where I'm going - it's not clear, from this context, what it means to say that an intervention worked, or that an organization improved. One thing I always tell clients is that the process of developing measures is important, and that you're going to be doing it over and over again. (I know, that's really two things.)
I don't think that there's an argument here for not trying to evaluate. But there is very good reason to be thoughtful about doing so.
The main focus of the article is the disparate data requests generated by funders - and often, each requires its own form for reports even if data are similar. I've had to gather and report this kind of data, and it can be a problem, not least because the process may not generate data that's useful for managing a program.
The article distinguished what it calls "summative" evaluations: did the intervention "work"? from formative evaluations: does the evaluation help the organization "improve"? You can see where I'm going - it's not clear, from this context, what it means to say that an intervention worked, or that an organization improved. One thing I always tell clients is that the process of developing measures is important, and that you're going to be doing it over and over again. (I know, that's really two things.)
I don't think that there's an argument here for not trying to evaluate. But there is very good reason to be thoughtful about doing so.
Subscribe to:
Posts (Atom)