Cities, adaptability, and climate change

The Atlantic's Cities blog ran a column last week called "Which Cities are Most Prepared for Climate Change?" It came to a pretty grim conclusion:

95 percent of major Latin American cities are actively planning for climate change, according to the report.

Canadian cities are also preparing themselves, with 92 percent of its major cities currently undertaking adaptation planning efforts. Similar preparations are being made in 80 percent of African cities, 84 percent of European cities and 86 percent of cities in Australia and New Zealand. Asian cities are less involved, with 67 percent reporting climate adaptation planning. And at the bottom of the list is the U.S., where only about 59 percent of major cities are actively preparing for the impacts of climate change.

Wow. Sounds as if we have some catching up to do. I clicked through to the source material, a 2012 report from ICLEI - Local Governments for Sustainability. According to ICLEI's website, it's a voluntary membership organization of government members devoted to sustainable development. OK, so far, so good. The website says that ICLEI has 1220 government members, representing 569,885,000 people.

But the methodology section of the survey raises some concerns. Check out this table:

Ten complete responses from Africa which has 29 member cities? That seems like a low response rate. What about the other cities in Africa?

And there's more - the survey was sent to 1171 communities that were members of ICLEI then - but the researchers had incorrect email addresses for 96 of them. 468 cities completed some or all of the questionnaire, but only 418 completed it fully.

Oh, and how big are the members? Some are quite big, but others are very small, 25,000. New York City and Los Angeles are on the list of member cities, but did they complete the questionnaire? It would be helpful to know how many large and how many small cities responded to the questionnaire.

The lesson? When you read someone's interpretation of survey results, it's important to think critically - even if they did not. In this case, spending a few minutes thinking about the representativeness of the survey respondents should have caused the reporter to dial back the conclusion.