advertisement

The “No Miracles Argument” and the “Base Rate Fallacy”1 In their (2004) Magnus and Callender introduce the distinction between what they call “retail” arguments for scientific realism, that is arguments about specific kinds of things such as neutrinos; and wholesale arguments, that is arguments about all or most of the entities posited in our best scientific theories (321). They conclude that wholesale arguments are hopeless – that “the great hope for realism and anti-realism lies in retail arguments that attend to the details of particular cases.” (336) For the most part they consider the so called “no miracles argument” and work by taking themselves to show that such an argument commits what is known as the “base rate fallacy”. Here I will agree with their pessimistic assessment of prospects for wholesale arguments but urge that the base rate fallacy functions in this discussion only to force a charitable restatement of the no miracles argument over its familiar statement. Since the “base rate fallacy” appears to have attracted some attention as a knock down case against the “no miracles argument” it will be worthwhile to get a clearer view of the relation of these two kinds of consideration. According to the no miracles argument it would be a miracle to get all the astonishingly accurate and unexpected predictions that we do from our successful theories if the theories weren’t true. So we are supposed to conclude that they are true, or at least approximately true. In particular we are supposed to conclude that the things to which such theories purport to refer must be taken to exist. We don’t need any fancy statistics to see the alleged problem with this argument. Consider the following exchange: Realist: It would be a miracle to have the occurrence of all the predictions of the atomic theory tabulated by Perrin if there were no atoms. So we must conclude that there are atoms and that their properties are pretty accurately described by the atomic theory. (19th century) Anti-realist: Frankly, the whole idea that there should be atoms, things so fabulously small that they could not possibly be seen, itself is just astonishing. I think it would be a greater miracle if there were atoms than the miracle of nature conspiring to produce these striking regularities without the mechanism of atoms. Choose your own poison. I’ll take the data without atoms as the less painful horn of this dilemma. Does this response show that the no miracles argument is a fallacy? Well perhaps, as originally stated. But with just a smidgen of charity we can take the intended form of argument to have been all along: If there is evidence that can be explained or predicted on the basis of a theory but is otherwise very surprising, and if it would be less 1 Thanks to Craig Callender, Branden Fitelson, and Peter Lipton for helpful correspondence. surprising for the theory to be true, then conclude that the theory is true.2 The foregoing exchange between the realist and antirealist is a qualitative rendering of the appeal to the “base rate fallacy” and the last paragraph gives the appropriate response. Here is the way it plays out with the statistics. (I follow the example used by Magnus and Callender.) Suppose there is a disease with a frequency in the population of .001, and a test for the disease that is certain to be positive if the subject has the disease but also exhibits a false positive rate of .05. If ‘h’ stands for the hypothesis that the person to whom the test is administered has the disease and ‘e’ for the result that the test is positive, we have P(h) = .001, P(e/h) = 1 and P(e/~h) = .05. Question: If someone tests positive, what is the probability that they have the disease? Apparently a lot of smart people will say (around) .95. But if 1000 people are tested, on average 1 will test positive because they have the disease and 50 will yield a false positive. So the rate of people who have the disease among those who test positive is very close to .02. It is claimed, now, that the same mistake is being made in the no miracles argument. Let’s continue to work with the example of the “atomic hypothesis”. Perrin tabulated an astonishing catalogue of verified predictions, e, of the atomic hypothesis, h. P(e/h) = 1, we may suppose. P(e/~h) on anybody’s story is going to be very low. It would be a “miracle” to have e if h were false. So we are supposed to conclude that h is true. But this case has the same form as the disease example. Ones intuition is that the evidence makes it likely that the subject has the disease/likely that the atomic hypothesis is true. In both cases the evidence does raise the probability of the truth of the hypothesis. But in the disease case this probability is still not high. So no reason has been given for concluding that the evidence makes the atomic hypothesis likely to be true. Let’s look at the structure of these arguments in a little more detail. Talk of “it would be a miracle if” is a metaphor that we can also more flexibly express with “it would be extremely surprising if”, that in turn is an attitude that one can describe with a probability function: 1/P(e) can be used as a measure of how surprising it would seem if e turned out to be true. Reexpression with a probability measure facilitates comparison with the medical example. ‘e’ describes a positive test result in the medical example and the successful evidence in the case of a scientific theory or theories. In the medical example ‘h’ names the hypothesis that the person tested has the disease. In the case of science, if we are concerned with a retail argument, ‘h’ names some specific scientific hypothesis or theory – we can think of the “atomic hypothesis” as a concrete example; or, if we are thinking about the wholesale version of the argument, h will refer collectively to our “mature” 2 I’ll leave it to the scholars to inspect the copious literature on the no miracles argument to determine whether or not this was really the intended argument all along. theories. In both cases we take P(e/h) to be very high – we can think of it as P(e/h) = 1. In both cases we also take P(e/~h) to be low. The fallacious thinking in the medical case goes like this. If h, e is very likely - it would not be at all surprising. If ~h, e is unlikely – it would be very surprising. If ~h, e would be, well maybe not a miracle, but not what one would have expected. So conclude h. This is an instance of a rule of inference that appeals to what are called likelihoods, the numbers P(e/h) and P(e/~h): If the likelihood ratio P(e/h)/P(e/~h) is high, conclude h. In the medical case it is easy to see how to rectify the situation, for it is easy to see that in every case if P(e/h) and P(~h) are both close to 1, P(h/e) is close to 1 iff P(e/~h) << P(h). In the medical case P(e/~h) > P(h), so P(h/e) is not close to 1. Earlier I suggested that this condition, or a qualitative analog, can be used to shore up statement of the no miracles argument: In discussing an antecedently unlikely theory (low P(h) ) and if one has evidence that is to be expected on the basis of the theory (high (Pe/h) but exceedlingly surprising otherwise (very low P(e/~h), and if in addition the degree of surprise assigned to the evidence on the assumption that the theory is false is much greater than the degree of surprise associated with the truth of the theory ( (P(e/~h) << P(h) ), conclude that the theory is true, or that it merits a high degree of belief. Indeed, once stated, I don’t think that excess charity is required to suppose that this last condition was tacitly intended all along by advocates of the no miracles argument. On this evaluation, it would hardly seem that the no miracles argument is an instance of the base rate fallacy, though comparison to cases in which the fallacy is committed shows us how better to state the argument. But once we recognize this refinement in statement we also see a further potential objection: The refined statement makes clear that the no miracles argument must appeal to P(h), the “base rate”, better known as the “prior probability” of the hypothesis, theory, or theories under consideration. How is this prior probability to be known? Magnus and Callender argue briefly, but cogently, that the case is hopeless for the no miracles argument on any interpretation of P that requires the use of statistics to justify the prior, P(h). What is the relevant reference class of hypotheses? How are the hypotheses to be individuated? Even if such questions could be answered, the statistics could be done only if we had some independent way of determining whether individual theories get into the reference class, that is whether they are true, likely to be true, or worthy of belief; and then the no miracles argument would be superfluous. (328). On any interpretation of P on which statistics would be required to justify the prior, any version of the no miracles argument presented as a wholesale argument will be hopeless. What I don’t think Magus and Callender make quite clear is that this conclusion holds just as well if the argument is applied retail. If we must rely on counting theories to determine the prior for the atomic hypothesis, what will the relevant reference class be? And how will we determine whether or not a theory qualifies by way of truth or high confirmation or failure thereof? So if there is any way for this kind of argument to get a purchase, the priors will have to be interpreted in the tradition of subjective degrees of belief, as a representation of the agent’s unjustified epistemic inclinations before the evidence is brought to bear, an option that Magnus and Callender acknowledge. (328) Many still reject any such appeal: “Crazy” priors will lead to crazy conclusions, so conclusions that are based on unjustified priors can’t, it is held, count as rational. But in this post foundationalist age most have acknowledged that epistemic arguments have to start from substantive unjustified premises. One must, at some stage, have unjustified prior opinions about how the evidence is to be brought to bear. Priors interpreted as subjectively held prior opinions are the Bayesian subjectivists’ way of incorporating this conclusion. On the original formulation of the no miracles argument there was no apparent appeal to priors. So, it would seem we must conclude, comparison with the base rate fallacy cases has forced a restatement on which we can see, in this regard, that the no miracles argument is weaker than it had previously appeared. I don’t think that such an evaluation is right. In the medical case we suppose that we have an objectively determined value for P(e/~h), the rate of false positives. But how is the value of P(e/~h) to be determined in the case of, say the atomic hypothesis, in the original formulation of the no miracles argument? What is the “correct” surprise value, degree of belief, or probability of the evidence on the supposition just that the atomic hypothesis is false? In the case of h a scientific hypothesis, theory, or range of theories, this value is as illusive as the prior, P(h), just as much a quantity underwritten only by the agent’s unjustified prior attitudes. This is the problem known among Bayesians as the problem of the “catchall” hypothesis, the problem of ones attitude towards the evidence on the assumption that all of the explicitly proposed ways of accounting for, predicting, or explaining the evidence should be false. The no miracles argument, as originally stated, had to appeal in this respect, to unjustified prior opinion. The need to appeal to unjustified prior opinion was really an element in the argument pattern all along. I conclude that the appeal to the base rate fallacy plays the limited role of forcing us to clarify the statement of the no miracles argument. Either one will reject a pattern of argument if it appeal to unjustified prior attitudes or not. If the former, one will reject the no miracles argument outright for its appeal to P(e/~h), or the qualitative counterpart, the prior opinion that without the truth of the theory under test the predicted evidence “would be a miracle”. And if the argument pattern is rejected outright, there is no need to appeal to the considerations of the base rate fallacy. Or, if unjustified prior attitudes are admitted, then in particular, appeal to the unjustified prior, P(h), is admissible. On this option consideration of the base rate fallacy has performed the useful work of showing that a proper statement of the no miracles argument requires appeal to P(h) as well as to P(e/~h), but not that it counts as a fallacy. For reasons Magnus and Callender give, when viewed as a statistical argument, whether retail or wholesale, the no miracles argument is a howler. But not by running afoul of the base rate fallacy. If unjustified priors are admitted, P(e/~h) and P(h) equally, none of the problems so far discussed apply. However, as Magnus and Callender briefly mention, on a subjectivist approach the wholesale/retail distinction can make an important difference: “In the present wholesale case, however, where the entire fate of realism or anti-realism seems bound up with the priors, we can’t imagine how one could find a reasonable set of priors.” (329). Strictly speaking, the question of unjustified priors is a private matter between agents and their subjective attitudes. But when the subject is all of “mature” science, the extent to which its predictions (most of which otherwise would never have come to light) would have been miraculous if all? most? much? of mature science were not to be true, most of us will see restricting appeal to our subjective attitudes to concrete, retail cases as a much more sound policy.