PH231 Evidence and Policy

Class 2: Evidence from Tests and
Probability Theory

Alexandru Marcoci

Extracting Theory from Evidence

"[T]here is no more rational procedure than the method of trial and error--of conjecture and refutation: of boldly proposing theories; of trying our best to show that these are erroneous; and of accepting them tentatively if our critical efforts are unsuccessful." (Popper 1963)

[A] theory of induction is superfluous. It has no function in a logic of science. The best we can say of a hypothesis is that up to now it has been able to show its worth, and that it has been more successful than other hypotheses although, in principle, it can never be justified, verified, or even shown to be probable. This appraisal of the hypothesis relies solely upon deductive consequences (predictions) which may be drawn from the hypothesis: There is no need even to mention 'induction'. (Popper 1932, 315)

The Inaudible 'Yes'

I do not think that we can ever seriously reduce by elimination, the number of the competing theories, since this number remains always infinite. What we do - or should do - is to hold on, for the time being, to the most improbable of the surviving theories or, more precisely, to the one that can be most severely tested. We tentatively ‘accept’ this theory - but only in the sense that we select it as worthy to be subjected to further criticism, and to the severest tests we can design. On the positive side, we may be entitled to add that the surviving theory is the best theory - and the best tested theory - of which we know. (Popper 1932, 419)

Poisson's Spot

"I wish to record my unbounded admiration for the work of the experimenter in his struggle to wrest interpretable facts from an unyielding Nature who knows so well how to meet our theories with a decisive No - or with an inaudible Yes." (Weyl in Popper 1932, § 85)

The Decisive 'No'

"[C]riteria of refutation have to be laid down beforehand: it must be agreed which observable situations, if actually observed, mean that the theory is refuted." (Popper 1963)

"In general we regard an intersubjectively testable falsification as final (provided it is well tested). [...] A corroborative appraisal made at a later date [...] can replace a positive degree of corroboration by a negative one, but not vice versa." (Popper 1932, § 82)

The 'Decisive' No

"Without theory it is impossible to regulate a single instrument or interpret a single reading." (Duhem 1954, 182)

"Our statements about the external world face the tribunal of experience not individually but only as a corporate body [...] Any statement can be held true come what may, if we make drastic enough adjustments elsewhere in the system. Even a statement very close to the periphery can be held true in the face of recalcitrant experience by pleading hallucination or by amending certain statements of the kind called logical laws. Conversely, by the same token, no statement is immune from revision." (Quine 1951, 42-3)

"[E]ven if Keplerian ellipses had refuted the Cartesian theory of vortices, only Newton's theory made us reject it; even if Mercury's perihelion refuted Newtonian gravitation, only Einstein's theory made us reject it. All that a refutation does is to enhance the problematical tension of the body of science and indicate the urgent need of revising it - in some yet unspecified way." (Lakatos 1968, 155)

"Falsificationism demands, therefore, that at least in a given critical situation, the body of science be divided into two, the problematic and the unproblematic." (Lakatos 1968, 157)

"If we perform an experiment, it depends on our methodological decision which theory we regard as the touchstone theory and which one as being under test; but this decision will determine in which deductive model we shall direct the modus tollens. Thus a 'potential falsifier', B, of T1, given some touchstone theory T2, may refute T1; but the same statement B, if regarded as a potential falsifier of T2, given T1 as touchstone theory, refutes T2." (Lakatos 1968, 157)

Remember: "[C]riteria of refutation have to be laid down beforehand: it must be agreed which observable situations, if actually observed, mean that the theory is refuted. "

The 'Decisive' No

Lakatos 1968, pp. 160-1

Prout's Law: All atoms are compounds of hydrogen atoms and thus the atomic weight of all pure (homogeneous) chemical elements are multiples of the atomic weight of hydrogen. (1815)

Berzelius's Refutation: Chlorine is a pure (homogeneous) chemical element and its atomic weight is 35.45 times that of hydrogen. (1828)

Prout's Law is cited as the inspiration for a series of experiments performed by Rutherford in 1917.

Harkins's Revival of Prout's Law: Chlorine was found to be composed of the isotopes Cl-35 and Cl-37, in proportions such that the average weight of natural chlorine was about 35.45 times that of hydrogen. For all elements, each individual isotope (nuclide) of mass number A was eventually found to have a mass very close to A times the mass of a hydrogen atom, with an error always less than 1%. (1925)

Remember: "In general we regard an intersubjectively testable falsification as final (provided it is well tested). [...] A corroborative appraisal made at a later date [...] can replace a positive degree of corroboration by a negative one, but not vice versa." (Popper 1932, § 82)

How do these clarifications inform your view with respect to how evidence is to be used in policy-making?

References

  1. Pierre Duhem. 1954/1991. The Aim and Structure of Physical Theory. Princeton University Press.
  2. Imre Lakatos. 1968. Criticism and the Methodology of Scientific Research Programmes. Proceedings of the Aristotelian Society 69: 149-186
  3. Karl Popper. 1932/2002. The Logic of Scientific Discovery. Routledge.
  4. Karl Popper. 1963. Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge.
  5. W.V.O. Quine. 1951. Two Dogmas of Empiricism. The Philosophical Review 60: 20–43.