Making Decisions with science

I first published this as a note on Facebook Dec 25, 9:49 PM
The other day a friend sent me a quote from the new president elect Obama - I can't recall the exact words but embedded in there were sentiments about getting back to science based decision making. Now, I'm the first to agree that something went seriously awry with how science was used by the Bush Administration, but I wonder about this mad dash to "use science to make decisions". Science can certainly be used to tell us the consequences of our decisions, but it seems to me that asking science to tell us which decision to make is fraught with hazards.

Imagine going to a financial planner and saying, "what should I invest in?". A scrupulous financial planner responds by asking you what your goals are - are you saving for a house? retiring in 5 years? 30 years? The reason these questions are important is that without knowing what your objectives are, the financial planner cannot help you choose from among the range of options. Each option - bonds vs. stocks say - has different consequences over different time frames, and which one is best depends on your objectives. If you want to retire in 5 years you want to protect your capital against events like the last 6 months or so of stock market activity! If you're taking a longer view, the volatility of stocks is offset by their long term average growth rates.

The point is, you set your objectives, and then the financial planner tells you what the consequences are of each different choice you can make. You still choose. You make the tradeoffs among objectives of growth and risk. The planner doesn't do that for you. Similarly, scientists can tell decision makers what the consequences of different choices are, but it should not be left to scientists to set objectives. That is the role of politicians and public debate. Given clear objectives, science can be of great service in predicting consequences. Unfortunately, that's often not what happens.

It is easy for a policy maker or politician, faced with a difficult natural resources conflict, to say "Oh, we should let the science sort it out". Best available science, that's the phrase in the endangered species act and other environmental legislation from the 70's. The difficulty with this strategy is that scientific predictions usually come with a certain amount of risk - think of weather forecasting - not a precise prediction, but good enough for making some sorts of decisions (like weather to take an umbrella or not). In addition to producing a distribution of possible outcomes, there is often greater uncertainty associated with the fundamental assumptions underlying any particular analysis. This uncertainty creates the opportunity for two different, but equally valid, scientific predictions about the outcomes of a decision. This kind of scientific conflict arises when science is asked to stand in for debates about the fundamental values and objectives of society, because each side in a conflict can make predictions that match their needs by selectively choosing assumptions. This ends up reducing the credibility of scientific predictions, and eroding the value of science in supporting decision making.

So what to do about this? I think the single most important thing is not to let our politicians off the hook. Demand that they debate the values that underlie decisions - and that they set objectives by which their decisions can be evaluated. Measurable objectives.

Of course, having measurable objectives doesn't guarantee making good decisions. A great example of this is the No Child Left Behind Act - plenty of objectives, but not well thought through consequences of the actions associated with the act. Frankly, insisting that politicians set objectives scares the living daylights out of me, because politicans don't appear, well, trustworthy just now. Unfortunately we live in a system that vests those particular people with the power to make decisions, and it seems to be a better system than the alternatives that have been tried over the millenia.

So the next time someone praises the use of science in decision making - ask them who's setting the objectives. That's the most important question we can ask.