Office of Scientific and International Cooperation

Abstract: Many widely used models amount to an elaborate means of making up numbers, with little connection to the real-world phenomena they purport to explain. But once a number has been produced, it tends to be taken seriously and its source (the model) is rarely examined carefully. Common steps in modeling to support policy decisions, such as putting disparate things on the same scale, may conflict with reality. For instance, not all costs and benefits can be put on the same scale; not all uncertainties can be expressed as probabilities; and not all model parameters measure the real-world quantity whose name they are given. Many uses of models in applied statistics lead to Type III errors: answering the wrong question. I will give examples from seismology, wind-turbine bird deaths, soccer penalty cards, gender bias in academia, and climate policy.