Measurement of intangibles at Harvard Business School

I read this article, published in the New York Times on September 7, about the administration's efforts to ensure gender equity at Harvard Business School with a great deal of interest. It's partly for the detailed glimpse into another world, and it's partly due to some of the great comments people made.

But during [graduation] week’s festivities, the Class Day speaker, a standout female student, alluded to “the frustrations of a group of people who feel ignored.” Others grumbled that another speechmaker, a former chief executive of a company in steep decline, was invited only because she was a woman. At a reception, a male student in tennis whites blurted out, as his friends laughed, that much of what had occurred at the school had “been a painful experience.”

The article is relevant here because of the focus of the administrators on measurement. Here's what they did:

* Turned what was subjective into an objective measure:  Women in the B School lagged in class participation - but participation grades are both subjective and dependent on memory. The business school administrators posted stenographers in every class so faculty no longer had to rely on their memories.

* Provided information quickly: As the reporter describes it, "[n]ew grading software tools let professors instantly check their calling and marking patterns by gender." Information was used to identify and change behavior, not to punish. In fact, it appears that the administrators pushed responsibility for change down to the front lines, making it a team effort. One professor stated that the message he got from the administration was: “We’re going to solve it at the school level, but each of you is responsible to identify what you are doing that gets you to this point.” Management trusted workers to get the message, and to change.

* Developed a theory and tested it: the article reports that an additional factor, one the school really couldn't control, was contributing to the women's minimal participation. Social success was as important as academic success, possibly more important, and class participation could hurt social capital.

As you have probably surmised, all this was expensive and time consuming. At least so far, the B School administrators haven't been able to try other approaches. And it's too soon to tell whether salaries for men and women will be comparable 10 years or more after graduation. The atmosphere, as reported, sounds a lot better, but, as the article points out, the experiment brought with it unintended consequences and helped other issues - like class differences - surface. The story's not done yet, but the tale so far makes for fascinating reading. (The graphics online are much better than they were in the physical newspaper.) And so do the comments.

Update: This post has been updated to correct a typo.

Harvard Business Review on measuring impact

Here's a a link to a thoughtful piece by Alnoor Ebrahim in the Harvard Business Review called "Let's Be Realistic about Measuring Impact." It compares the measurement processes of three different organizations, each of which uses a different approach. One measures outputs, another measures short-term outcomes and estimates long-term outcomes, while the third thinks hard - and waits for - long-term outcomes. Each uses some form of proxy measures when necessary. Each organization puts the resources necessary into making its measurements meaningful. And they've figured out a way to get around, where necessary, their research limitations. Says Ebrahim:

Notice that none of these three organizations typically measures impact directly. They hypothesize what the outcomes and impacts might be but only in some instances are they able to follow through by commissioning their own research or multi-year evaluations. And these are sophisticated funders and investors who are much better positioned to measure long-term results than the front line organizations that contend with funding shortages and operational challenges every day.

I've written before about the importance of using outcome measures. It's hard to do well, but well worth the effort. Has your organization tried? What were the problem areas? What worked well?

Extreme weather and ocean salinity

Illustration from Climatecentral.org

Climate Central has posted a very clear explanation of the new research using changes in ocean salinity to stand in for worldwide drought and rainfall measures. The idea is straightforward: with climate change, extreme weather cycles are likely to worsen. We're starting to see that regionally, but regions - even ones as large as a continent - are too small to give a global picture. Hence the move to measure ocean salinity. (No one has rainfall data over the ocean, so salinity serves as a proxy measure.) As you can see from the map above, ocean saltiness is increasing. And what's really news is that it appears that the change is happening about twice as fast as the land-based predictions had suggested.