"The Half-Life of Facts," by Samuel Arbesman

In Samuel Arbesman's expression, set out in his new book "The Half-Life of Facts: Why Everything We Know Has an Expiration Date," we dwell in uncertainty. Things around us, things we think we know, are always changing. Sometimes it's because we learn more - many of Arbesman's examples come from our increasing knowledge of dinosaurs (they didn't have feathers or colors, and barely had brains, when I was a kid). Sometimes we get better at measuring - look at what happened to the size of Pluto (it is smaller than we were taught and is now no longer thought to be a planet). The rapid pace of change can feel confusing, if not overwhelming. How can we possibly keep up?

In his clear and well-written book Arbesman tells us how to approach the problem. He demonstrates how knowledge changes in regular ways, ways that are systematic enough for us to understand. He argues that understanding the changes helps us make sense of the world and allows us to prepare for the changes we can anticipate. (I think these are what have been called in another context the known unknowns.) From my blog's perspective, it's a well-written, useful book about the importance of thinking critically.

Arbesman uses the work of John Ioannidis to remind us that one study is never enough, and that we are prone to jumping to conclusions. Just because something has been published doesn't mean that it is true, or will hold up over time. In fact, chances are, it won't. (As in most other statistical situations, that means the aggregate of papers, because you can't at any particular time tell which ones will last.) Replication is the best way to test scientific findings, but replicating someone else's work doesn't mean prizes or even necessarily publication. Here are a few common-sense corollaries that Arbesman provides to keep in mind when you read or hear about new studies:

• The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.
• The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.
• The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.
• The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.
• The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.
• The hotter a scientific field (with more scientific teams involved) the less likely the research findings are to be true.

It's equally important to think critically about the smaller things in our lives too. We communicate through networks, of both the interpersonal and the electronic kind. Unfortunately, we also transmit errors that way. Errors, Arbesman tells us, often spread more slowly than truths, but they also linger. His solution is one well worth adopting: be critical before you push send or post or otherwise share that scary story. (He reminds us that there are a couple of websites that provide the research, including snopes.com and xkcd.com, so that we don't have to worry about debunking each myth that comes our way.)
There's a lot more going on in this entertaining and interesting book. If you think, as I did, that the title is a metaphor, well, nope. Growth in human knowledge is exponential, though different fields have different rates of increase. New discoveries prove old ones wrong, and what we know changes. There are a couple of concomitants to this information. First, it's getting harder to make discoveries (though contiguity with one's collaborators makes everyone's work better.) Second, over time, most scientific papers will be superseded or shown to be wrong. The turnover is true for central tenets as well as for details.

You may have heard of Moore's Law, the one that says that the capacity of a computer chip doubles every 12-18 months. In a chapter titled "Moore's Law of  Everything" Arbesman generalizes that tenet to growth in many areas. The interplay between science (what we know) and technology (what we can do) depends on the growth of knowledge. But it also depends on the growth of the human population: more population means more knowledge. Of course, he adds, the people need to be interested in and able to become scientists and engineers, and we need to be able to communicate what we have learned. Sometimes knowledge in one field stays hidden from experts in another, but we are beginning to understand the mechanisms that allow the systematic exploitation of one area's learning in another. We often study what interests us, what we like, or what’s easier to discover. And cognitive biases can interfere with our ability to understand what is right under our noses.

This is not a book of philosophy or statistics but a very good effort at making some useful work accessible. Do you agree? Let me know what you think in the comments.

Cognitive bias and individual stock traders

Repeating yesterday's theme of cognitive bias, here's an article in the "Deal Professor" column of the New York Times illustrating herd behavior and loss aversion among individual investors. Steven M. Davidoff writes of individual investors that a study

found that the 20 percent who traded most actively earned 7 percentage points a year less than the buy-and-hold investors, the 20 percent who traded least actively. For the individual investor, that can add up to hundreds of thousands of dollars over a lifetime.

This is not surprising. Even mutual fund managers have trouble beating the market. Last year, according to S.& P. Indices, 84 percent of actively managed funds did not beat the Standard & Poor’s index representing that fund’s sector. Going back over five years, 61 percent of funds underperformed. Even so, most mutual funds beat individual investors who try to do it themselves.

If the professionals have such problems, individual investors don’t have a chance. They are not as knowledgeable and not as disciplined. Study after study has found that individual investors have a disposition effect — that is, they tend to sell winners too soon and hold on to the losers by refusing to recognize their failure.

Cognitive biases and strategic decision-making

Daniel Kahneman's book "Thinking, Fast and Slow" synthesizes a great deal of research over the past several decades about the brain's thinking and decision-making processes. It's a great book, well worth reading. But it's pretty long. This article from McKinsey classics, "Hidden Flaws in Strategy," is nearly 10 years old, but is worth reading for its still-valid insights. (It's free after registration.) The article looks at common cognitive biases in decision making and suggests ways to avoid them.

1. Overconfidence/overoptimism - we tend to look at the bright side, and wildly overestimate our abilities to predict. To counter this tendency, the authors advise testing strategies under a wide range of scenarios, taking the most pessimistic scenario and making it worse, and ensuring that you have the capacity to be flexible as uncertainties resolve.

2. Mental accounting - we all put some spending into categories that saves us from having to look at it too closely. The authors recommend adherence to "a basic rule: that every . . . dollar . . . is worth exactly that, whatever the category. In this way, you will make sure that all investments are judged on consistent criteria . . . "

3. Don't be too wedded to the status quo, but be prepared to stick with it when it's the better choice. How to tell? The authors recommend two approaches: a) Take a radical view of your entire portfolio of programs and consider closing or changing all of them; and then b) Analyze your status quo options the same way you would change options. "Most strategists are good at identifying the risks of new strategies but less good at seeing the risks of failing to change."

4. Anchoring - Our brains tend to stick with, or anchor, to a suggested number, whether it is relevant to whatever we've been asked about or not. Sellers might use the tendency to their advantage during negotiations or advertising. But the tendency can impair decisions. Put comparisons in a larger context: 20-30 years, for example.

5. The sunk-cost fallacy - loss aversion and anchoring often lead us to continue an investment even after it has turned sour. To avoid it, the authors say, look at each incremental investment separately, with a fully analysis. Be ready to end experiments early. And condition further funds on meeting certain targets.

6. Know when to follow the herd - and when to go your own way. Good strategies often break away from a trend, the authors say. Combined with the principle of ending experiments early, it's can be smart to disregard the received wisdom.

7. Know when to get excited. OK, the authors don't quite put it this way, but a wise woman I once worked for did. Sometimes waiting and seeing is the best policy.

8. Make sure your consensus, when you have one, is real. False consensus can be reached when a strong leader thinks she has sought and received objective counsel but for whatever reason (they can include pressure to agree, selective recall, confirmation bias, or a biased evaluation) the consensus is a false one. To minimize the risk, the authors say, make sure your culture values challenges and open criticism. In addition, make sure the strong players have checks and balances so that they can't simply dismiss challenges to their proposals without reviewing them. And, as I said yesterday, make sure you search for as many reasons not to do something as you can come up with for a reason to do it.