How scientific models both help and deceive us in decision making

07 Jun 2023

Michela Massimi reviews Escape from Model Land by Erica Thompson

Human insights Given the inherent limitation to scientific models of, say, climate change or the spread of viruses, it could make sense to get a wider range of people involved when turning the predictions from such models into public policy decisions. (Courtesy: iStock/FotoMaximum)

We live in a society where scientific models surround us. They are used for everything from creating weather bulletins and making climate projections to providing economic forecasts and informing policies for public health. But despite being such useful tools, all scientific models have limitations. Because as any modeller knows, the output of a model is only as good as the data you put in.

What’s more, uncertainties creep in at every corner of the modelling exercise. The results of a model depend on, for example, the values of the parameters, the boundary conditions, and the basic assumptions of the model itself. So how can we ensure scientific models are used responsibly when deciding matters of public policy? That’s the question tackled in Escape from Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It by Erica Thompson, who trained as a physicist and is now in the Department of Science, Technology, Engineering and Public Policy at University College London.

Erica Thompson’s book is a tour de force in explaining the practical challenges of scientific modelling

Thompson’s book is a tour de force in explaining the practical challenges of scientific modelling. What happens, Thompson asks, if the data we compare our model against are scant or hard to harvest? How can we assess the reliability of long-term model projections? And how can we work out if a model is a good representation of the real world? These are important questions because if we want to escape “model land”, we have to see where the limits of modelling lie.

Think, for example, how politicians used epidemiological models during the COVID-19 pandemic. By seeing what might happen if nothing were done to stop the spread of the virus, governments used these worst-case-scenario forecasts to justify lockdowns and policies on social distancing. Or think about how we decide on climate policy by looking at long-term projections of what might happen with different levels of greenhouse gases in the atmosphere.

But what exactly lies beyond model land? Thompson’s bold vision is that we should empower humans more wisely, deploying our expert judgement to use models reliably when making decisions. We can, the author argues, make models more trustworthy by being transparent about our value judgements, declaring where our conflicts of interest lie and involving a greater variety of experts.

“If we are serious about addressing lack of confidence in science,” Thompson writes, “it is necessary for those who currently make their living from and have built their reputation on their models to stop trying to push their version of reality on others.” In particular, the author believes we should encourage modelling efforts from under-represented groups and those with different political views. “[We should] acknowledge that decision-making requires value judgments as well as predicted outcomes. And yes, that’s a big ask.”

As such, the book builds on a well-established tradition in the contemporary philosophy of science, which examines how our own human values enter science when interpreting and selecting data, when choosing which approach to adopt to a problem, and when interpreting the outcomes of models. Even the latest 2022 report from the Intergovernmental Panel On Climate Change (IPCC) contains references to the philosophical literature.

Thompson believes that the difference between the outcome of a model and the actions we take based on it – what the author dubs the “accountability gap” – can be bridged by offering “an expert bird’s-eye perspective from outside Model Land”. So rather than just reporting the results of models, we should “offer additional expert judgements, arrived at by consensus, about the degree to which model results are judged to be reliable”.

The author believes that we need a rich and diverse variety of expert voices when extrapolating from models for decision-making

The author essentially believes that we need a rich and diverse variety of expert voices when extrapolating from models for decision-making. Scientific models, in other words, aren’t just devices that take snapshots of some well-defined piece of reality. As the University of Edinburgh sociologist Donald MacKenzie says, we should see them as “engines” that take an active part in the decision-making process.

In my own recent book Perspectival Realism, I discuss how scientific models deliver knowledge of what is possible by acting as what I call “inferential blueprints”. Models allow different communities to come together and make relevant and appropriate inferences about a target system. The Coupled Model Intercomparison Project, for example, doesn’t just involve modellers but also includes dendroclimatologists and scientists studying isotopes in corals, who provide data to help us reconstruct how the Earth’s temperature varied in the past.

READ MORE

Model independence

Escape from Model Land draws on research carried out by David Tuckett from University College London, who has studied how people make decisions under conditions of “radical uncertainty” (i.e. when the uncertainty cannot be quantified). Thompson explains how models can help us to assess risks and make appropriate decisions even though our emotional attachment often makes us unwilling to alter our assumptions or take into account conflicting information or external views. That’s the reason behind Thompson’s call for diversity in modelling: it’s so we can improve our decision making and get better policy outcomes.

Overall, the author does a brilliant job at presenting technical information in an accessible and easy-to-read way. I found the book’s analysis of scientific modelling clear and well informed by the latest developments in philosophy. If we are truly to escape model land, as Thompson hopes, then we as humans – with all our biases and various levels of expertise – will have to be centre stage. Diversity, equality and inclusion will be crucial if models are to become more reliable and more trustworthy and, ultimately, allow us to make better and more informed decisions.2022 Basic Books 247pp £20/$30hb

Michela Massimi is professor of philosophy of science at the University of Edinburgh, UK, and is affiliated with the Higgs Centre for Theoretical Physics. She is also the author of Perspectival Realism (Oxford University Press 2022) and is the current president of the Philosophy of Science Association, e-mail michela.massimi@ed.ac.uk, Twitter @p_realism

FROM PHYSICSWORLD.COM    2/9/2023