Evidence and Policy

Welcome to the website for the PH231 class I taught at LSE in Lent 2015 for John Worrall. This website complemented the course's Moodle page and during term time it was updated weekly. Some of the resources here have been added after the last meeting in the hope that at some point I will have the opportunity to teach a variation of this course again. If you encounter any problems or if you have recommendations for improvement please get in touch.

The table below offers an overview of the course. Full bibliographic references can be found on this website in the "Readings" section. The assignments have to be submitted using the forms available in the "Assignments" section.

ClassDateTopicReadingsAssignmentResources
1 16 January The Theory of Evidence Dick Taverne. The March of Unreason (Prologue & Ch. 1)
John Worrall. A Bridge Over Troubled Cultures
Gerd Gigerenzer et al. Simple tools for understanding risks
Gerd Gigerenzer et al. Helping Doctors and Patients...

Assignment 0 Lecture 1
2 23 January Evidence from Tests
and Probability Theory
John Worrall. Probability Theory (Sections 1 & 3)
Karl Popper. Conjectures and Refutations. Ch. 1
Imre Lakatos. Science and Pseudoscience

Assignment 1 Lecture 2
Class 2
Handout 2
3 30 January Foundational issues about
significance tests (I)
John Worrall. Notes on Testing Statistical Theories
Howson and Urbach. Scientific Reasoning. Ch. 5
Gerd Gigerenzer. The Superego, the Ego, and the Id

Assignment 2 Lecture 3
Class 3
Handout 3
4 09 February Foundational issues about
significance tests (II)
John Worrall. Notes on Testing Statistical Theories
Howson and Urbach. Scientific Reasoning. Ch. 5
Gerd Gigerenzer. The Superego, the Ego, and the Id
Ronald Fisher. Statistical Methods and Scientific Induction
Kalinowski and Fidler. Interpreting Significance

Assignment 3 Lecture 4
Handout 4
5 13 February Clinical Trials and "Equipoise" John Worrall. Evidence and Ethics in Medicine
Emmanuel et al. What Makes a Clinical Trial Ethical?
Miller and Brody. A Critique of Clinical Equipoise
Angell. The Ethics of Clinical Research in the Third World
Clinical Equipoise
Assignment 4 Lecture 5
Handout 5
6 20 February What’s So Great About
Randomization? (I)
John Worrall. What Evidence in Evidence-Based Medicine?
Leonard Leibovici. Bloodstream Infection: Randomised Controlled Trial
Colin Howson. Hume's Problem, Ch. 3

Assignment 5 Lecture 6
Handout 6
7 26 February What’s So Great About
Randomization? (II)
John Worrall. Notes on Testing Statistical Theories
A. B. Hill. The Environment and Disease: Association or Causation?
John Worrall. Causality in Medicine: Getting back to the Hill top

Assignment 6 Lecture 7
Handout 7
8 06 March External Validity Adam La Caze. The Challenge of External Validity. Chapter 5
Nancy Cartwright. Evidence-based policy: what's to be done about relevance?
John Worrall. Evidence: Philosophy of Science Meets Medicine
Angus Deaton. Policy Implications Of The Gradient Of Health And Wealth

Assignment 7 Lecture 8
Handout 8
9 13 March From Evidence to Decision:
taking a 'wide screen' view
and avoiding 'precaution'
John Worrall. Do We Need Some Large, Simple Randomised Trials in Medicine?
Richard Peto et al. Large-scale randomized evidence
Cass Sunstein. Beyond the Precautionary Principle

Formative Report
Due: 13 March, 23:59
Lecture 9
Handout 9
10 20 March Some Loose Ends:
EBM and Bayesianism
John Worrall. Evidence in Medicine and Evidence-Based Medicine
B. Hurwitz. How Does EB Guidance Influence Determinations Of Medical Negligence?

Assignment 8 Lecture 10
Handout 10
3 May, 23:59 Summative Report
8 May, 10-11 Revision & Exam Preparation

Readings will be added one week before the class for which you have to prepare them. Additional further readings are listed in the "Additional Resources" section. The "Essays" section contains information on identifying other relevant materials. Please contact me if you have problems accessing any of the below.

Required Readings
  1. Dick Taverne. 2005. The march of unreason : science, democracy, and the new fundamentalism. Oxford University Press. Prologue and Chapter 1
  2. John Worrall. A Bridge Over Troubled Cultures: The Impact of Philosophy of Science in Great Britain. Unpublished
  3. Gerd Gigerenzer, Adrian Edwards. 2003. Simple tools for understanding risks: from innumeracy to insight. British Medical Journal 327:741-744
  4. Gerd Gigerenzer, Wolfgang Gaissmaier, Elke Kurz-Milcke, Lisa M. Schwartz, and Steven Woloshin. 2007. Helping Doctors and Patients Make Sense of Health Statistics. Psychological Science in the Public Interest 8:53-96
Further Readings
  1. Dick Taverne. 2005. The march of unreason : science, democracy, and the new fundamentalism. Oxford University Press
  2. Oliver Gillie. 2004. Sunlight Robbery: Health benefits of sunlight are denied by current public health policy in the UK. Health Research Forum
  3. Odette Wegwarth, Gerd Gigerenzer. 2011. "There is nothing to worry about”: Gynecologists’ counseling on mammography. Patient Education and Counseling 84:251-256
  4. Colin Howson. 2000. Hume's Problem: Induction and the justification of belief. Oxford University Press
Required Readings
  1. John Worrall. Probability Theory. Unpublished. Sections 1 and 3
  2. Karl Popper. 1963. Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge. Chapter 1
  3. Imre Lakatos. 1978. The Methodology of Scientific Research Programmes: Philosophical Papers Volume 1. Edited by John Worrall and Gregory Currie. Cambridge University Press. Introduction
Further Readings
  1. John Worrall. Probability Theory. Unpublished. Section 2
  2. Colin Howson and Peter Urbach. 1989/1993. Scientific Reasoning: The Bayesian approach. Second Edition. Open Court. Chapter 2
Required Readings
  1. John Worrall. Notes on Testing Statistical Theories. Unpublished.
  2. Colin Howson and Peter Urbach. 1989/2006. Scientific Reasoning: The Bayesian approach. Third Edition. Open Court. Chapter 5
  3. Gerd Gigerenzer. 1993. The Superego, the Ego, and the Id in Statistical Reasoning. In A handbook for data analysis in the behavioral sciences. Hillsdale, NJ: Erlbaum, pp. 311-339.
Further Readings
  1. Stephen Ziliak and Deirdre McCloskey. The Cult of Statistical Significance. Presented at the Joint Statistical Meetings, Washington DC, 3 August 2009
Required Readings
  1. John Worrall. Notes on Testing Statistical Theories. Unpublished.
  2. Colin Howson and Peter Urbach. 1989/2006. Scientific Reasoning: The Bayesian approach. Third Edition. Open Court. Chapter 5
  3. Gerd Gigerenzer. 1993. The Superego, the Ego, and the Id in Statistical Reasoning. In A handbook for data analysis in the behavioral sciences. Hillsdale, NJ: Erlbaum, pp. 311-339.
  4. Ronald Fisher. 1955. Statistical Methods and Scientific Induction. Journal of the Royal Statistical Society. Series B (Methodological).17: 69-78
  5. Pawel Kalinowski and Fiona Fidler. 2010. Interpreting Significance: The Differences Between Statistical Significance, Effect Size, and Practical Importance. Newborn & Infant Nursing Reviews 10 (1): 50–54
Further Readings
  1. Stephen Ziliak and Deirdre McCloskey. The Cult of Statistical Significance. Presented at the Joint Statistical Meetings, Washington DC, 3 August 2009
Required Readings
  1. John Worrall. Evidence and Ethics in Medicine. Unpublished.
  2. E.J. Emmanuel et al. 2000. What Makes a Clinical Trial Ethical? JAMA 283(20): 2701-2711.
  3. F. G. Miller and H. Brody. 2003. A Critique of Clinical Equipoise: Therapeutic Misconception in the Ethics of Clinical Trials. The Hastings Center Report 33(3): 19-28.
  4. M. Angell. 1997. The Ethics of Clinical Research in the Third World (Editorial). NEJM 337(12): 847-849.
  5. Clinical Equipoise. Wikipedia.
Required Readings
  1. John Worrall. 2002. What Evidence in Evidence‐Based Medicine? Philosophy of Science 69 (3): 316-330
  2. Leonard Leibovici. 2001. Bloodstream Infection: Randomised Controlled Trial. BMJ: British Medical Journal 323 (7327):1450-1451
  3. Colin Howson. 2000. Hume's Problem: Induction and the justification of belief. Oxford University Press. Chapter 3, pp. 48-51
Further Readings
  1. John Worrall. 2007. Evidence in Medicine and Evidence-Based Medicine. Philosophy Compass 2 (6): 981–1022
  2. Adam La Caze. 2008. The Challenge of External Validity. PhD thesis, University of Sydney. Chapter 3
  3. A. M. Thornett, S. Hettiaratchy, C. Hemsley, J. Hopkins, M. J. Brownnutt, C. I. Price, M. Lagnado, S. A. Schwartz, S. L. Black and L. Leibovici. 2002. Effect of Retroactive Intercessory Prayer. BMJ: British Medical Journal 324 (7344):1037-1039
  4. John Worrall. 2010. Evidence: Medicine meets Philosophy of Science. Journal for the Evaluation of Clinical Practice 16(2):356-362
Required Readings
  1. John Worrall. Notes on Testing Causal Theories File. Unpublished
  2. A. B. Hill. 1965. The Environment and Disease: Association or Causation? Proceedings of the Royal Society of Medicine 58 (5): 295–300
  3. John Worrall. 2011. Causality in medicine: Getting back to the Hill top. Preventive Medicine 53: 235-238
Further Readings
  1. David Papineau. 1994. The Virtues of Randomization. British Journal for the Philosophy of Science 45 (2): 437-450
  2. John Worrall. 2007. Why There's No Cause to Randomize? British Journal Philosophy Science 58 (3): 451-488
  3. Nancy Cartwright. 1979. Causal Laws and Effective Strategies. Nous 13 (4): 419-437
  4. Christopher Hitchcock. 2012. Probabilistic Causation. The Stanford Encyclopedia of Philosophy (Winter Edition), Edward N. Zalta (ed.)
Required Readings
  1. Adam La Caze. 2008. Evidence-based Medicine: Evolution, Revolution or Illusion? PhD thesis, University of Sydney. Chapter 5
  2. Nancy Cartwright. 2009. Evidence-based policy: what’s to be done about relevance? Philosophical Studies 143 (1): 127-136
  3. John Worrall. 2010. Evidence: Philosophy of Science meets Medicine. Journal for the Evaluation of Clinical Practice 16(2):356-362
  4. Angus Deaton. 2002. Policy Implications Of The Gradient Of Health And Wealth. Health Affairs, 21(2):13-30
Further Readings
  1. Nancy Cartwright and Jeremy Hardie. 2012. Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford University Press, pp 57-83
  2. G.W. Bohrnstedt and B.M. Stacher (eds.). 2002. What we have learned about class-size reduction in California. Sacramento, CA: California Department of Education
Required Readings
  1. John Worrall. 2009. Do We Need Some Large, Simple Randomised Trials in Medicine? In Suarez, Dorato & Redei (eds) EPSA Philosophical Issues in the Sciences, Vol 2, 289-302
  2. Peto, Collins and Gray. 1995. Large-scale randomized evidence: Large, simple trials and overviews of trials. Journal of Clinical Epidemiology 88 (1): 23-40
  3. Cass Sunstein. 2003. Beyond the Precautionary Principle. U of Chicago, Public Law Working Paper No. 38
Further Readings
  1. Dick Taverne. 2005. The march of unreason: science, democracy, and the new fundamentalism. Oxford University Press, Chapter 7
Required Readings
Further Readings
  1. Colin Howson and Peter Urbach. 1989/1993. Excerpts from Scientific Reasoning: The Bayesian approach. Second Edition. Open Court
  2. Michael Rawlins. 2008. De testimonio: on the evidence for decisions about the use of therapeutic interventions. The Lancet 372: 2152–2161

Please submit short and concise answers to all questions before the deadlines indicated below. [All assignments have now been closed. Please see Overview for further details.]

Assignment 0 was supposed to be submitted at the beginning of our first class.
Assignment 1 is now closed.
Assignment 2 is now closed.
Assignment 3 is now closed.
Assignment 4 is now closed.
Assignment 5 is now closed.
Assignment 6 is now closed.
Assignment 7 is now closed.
Assignment 8 is now closed.

SEP
Stanford Encyclopedia of Philosophy

The first port of call. In-depth articles written by experts in the field which you may use as references in your written work. Each entry contains an extensive literature to guide your further readings.

PhilPapers
PhilPapers

Search engine for papers and books in philosophy and cognate disciplines. The best way to search by keywords for extra literature for your essays and assignments. It also supports search by sub-fields.

GoogleScholar
Google Scholar
Academic search engine. Less useful than PhilPapers for searching by keyword but excellent for in-depth searches. Search for an article/book and then click on "Cited By" to find an extensive list of works engaging with it.

Wikipedia
Wikipedia

Not a reliable resource for philosophical literature, but very good for technical notions. I encourage you to explore the pages on probability theory. However, entries are not peer-reviewed so do not treat them as references but as guides for further reading.

Compass
Philosophy Compass

Similar in style to the Stanford Encyclopedia, the Philosophy Compass "covers the entire discipline of philosophy and publishes original, peer-reviewed, state-of-the-art surveys of current research." You can reference it in your own work.

Wolfram
Wolfram Alpha

A search engine with numerous functionalities, I recommend using it in tackling with the mathematics and statistics you encounter in your readings for this course. Also useful for checking your own calculations and mathematical/statistical hunches.

There are numerous excellent guides to writing an academic essay in philosophy. I strongly recommend consulting (at least) the following.

Pryor
Guidelines on Writing a Philosophy Paper

The standard guide to a good essay. One of the most comprehensive selection of tips on writing philosophy.

Roberts
7 Steps to a Better Philosophy Paper

Written by our very own Bryan Roberts, short and to the point. An excellent quick guide to consult before every essay you write.

Mendelovici
A Sample Philosophy Paper

A sample philosophy essay with margin comments by Angela Mendelovici.

Hargrave
How to Fail Philosophy Exams

A short guide on how to prepare for a philosophy exam and what happens if you don't.

Roberts
On Writing a Dissertation

Bryan Roberts has put together this very informative guidebook on writing an MSc Dissertation. The advice he offers, I think, is more general and it (at least partially) applies to your essays and assignments.

I will assess your written work using the following two feedback forms. Please have a look at the criteria mentioned and at the common mistakes listed under each of them. Use these forms to revise your assignments and essays before submitting.

Please read the scenario below very carefully and complete the task assigned.

HomeopathyYou are in charge of Administration at a large GP Practice. The GPs who run the practice are considering introducing a homeopathic service for their patients (which the NHS allows them to do, albeit as a private, i.e. patient-funded, service rather than as paid for by the NHS). They ask you to prepare a document summarising the pro’s (if any) and con’s (if any) of this proposal. As further background: one of the GPs is entirely dismissive of homeopathy on the grounds of the ‘craziness’ (his word) of its alleged physical basis; one has read a report of an RCT which ‘shows’ that homeopathy can outperform placebo and is inclined to be sympathetic; and another has said that she ‘doesn’t care about the evidence, so long as it [homeopathy] works’ and she points out that she has had a good deal of success with ‘placebo’ treatments for some conditions.

Task: Write a report that will form the basis for the discussion of the group, leading to a decision on whether or not to offer the homeopathic service. Your report should - without addressing your colleagues directly - address the concerns they have raised.
In writing your report you should make use of the literature suggested throughout the course and/or further literature which you identify as relevant. However, you should follow your own 'reading trail'. A useful starting point would be the statement of the NHS's current stance on homeopathy available here. You will be resubmitting a revised version of this piece of work as your summative essay for this course (naturally you are allowed, indeed encouraged, to take the feedback on the formative into account in completing the summative work). For the summative piece the length limit is 2500 words - if you exceed this your mark will be affected by a penalty. You should therefore try to stick to that limit in your formative work - though up to 3000 words would be acceptable.

Please include your full name and word count on the first page of your report and submit it in .doc, .docx or .odt format by email to A.Marcoci at lse.ac.uk before Friday, 13 March, 23:59. I will return your reports with extensive feedback in Week 10 before our class.

For the Summative Report you will have to resubmit a revised and anonymised version of your Formative Report. The deadline is 23:59 on Sunday, May 8 (the end of the first week of Summer Term). This is an official part of the examination and your submission should be made through Turnitin, following the instructions below. The word limit is 2500 words. This includes footnotes but not references. In order to promote equity penalties are imposed for exceeding the word limit.

On this page you can find links to all the materials used in class by both John and myself. Links will be updated weekly so please check back regularly.

Lecture Slides

Class Slides

Class Handouts

Weekly Assignments

Practice Exam Questions

Office Hours

I hold weekly office hours during term time (including ST) in LAK 1.02 on Fridays from 13:00 to 15:00. If you cannot make them please contact me and we will arrange a meeting outside of that interval. Otherwise, book an appointment via LSE4U.

You are also always welcome in John's office hours. To see him you have to book an appointment via LSE4U.

Error Statistics
Evidence-Based Policy Blog
Deborah Mayo is keeping a blog on philosophy of statistics called Error Statistics Philosophy. There are many entries relevant to the topics we cover in this course and there are extensive bibliographical recommendations. She also runs a label for Evidence-Based Policy.

Hargrave
Using Probability in the Courtroom

The English Court of Appeal questioned the nature of proabability: "When judging whether a case for believing that an event was caused in a particular way is stronger that the case for not so believing, the process is not scientific [...] and to express the probability of some event having happened in percentage terms is illusory." Coverage of a recent similar ruling.

Framing
On Framing Effects and Decision-Making

You saw that probabilistic judgements are sensitive to how the information is presented. The sensitivity of decision-makers to the way evidence is framed is a well-documented one. One of the seminal papers on framing is Daniel Kahneman and Amos Tversky. 1981. The Framing of Decisions and the Psychology of Choice. Science 211: 453-458.

MMR
MMR Panic: 15 Years After

Coverage of the effects of the (now retracted) paper connecting MMR and autism. Also, here is a short piece on the recent measles outbreak at Disneyland, CA.

Carnap
On the Verifiability Criterion of Meaning

In Chapter 1 of Conjectures and Refutations, Popper contrasts his criterion of demarcation to the logical positivists' verifiability criterion of meaning. If you would like to read more about the latter I encourage you to read Rudolf Carnap. 1932. The Elimination of Metaphysics Through Logical Analysis of Language. Erkenntnis 2:60-81.

Lakatos
Methodology of Scientific Research Programmes

In class we only discuss Lakatos very briefly. If you find his views compelling I encourage you to consult his 1968. Criticism and the Methodology of Scientific Research Programmes. Proceedings of the Aristotelian Society 69: 149-186 (especially for its very illuminating analysis of Popper) and his 1970. History of Science and Its Rational Reconstructions. Proceedings of the Biennial Meeting of the Philosophy of Science Association 1970: 91-136.

Goodman
On the (New) Riddle of Induction

In Chapter 1 of Conjectures and Refutations, Popper argues against Hume's psychological justification of induction. If you find the topic of induction interesting and wish to read more about its problems, Nelson Goodman. 1955. The New Riddle of Induction. Chapter 3 of his Fact, Fiction, and Forecast. Harvard University Press is one of the classics of analytic philosophy.

Popper
The Logic of Scientific Discovery

For a more in-depth understanding of Popper's views on falsifiability and induction see his 1932/2002. The Logic of Scientific Discovery. Routledge. Part II. Sections 3 and 4.

stats
Statistics as a Social Phenomenon

"Statistical rituals largely eliminate statistical thinking in the social sciences." Gerd Gigerenzer investigates the "null ritual" in his 2004. Mindless Statistics. The Journal of Socio-Economics 33: 587–606.

stats
The Cartoon Guide to Statistics

If you are looking for an alternative introduction to statistics you could try Larry Gonick's. 1993/2000. The Cartoon Guide to Statistics. Fourth Edition. William Morrow Paperbacks.

Interpretations
The Interpretations of Probability

In class we only discuss the formal notion of probability. However, if you want to understand the debates surrounding the use of probabilities, you need to understand better the difficulties with assigning meaning to it. A comprehensive overview can be found in Donald Gilles. 2000. Philosophical Theories of Probability. Routledge.

Monty Hall
The Monty Hall Problem

"Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, 'Do you want to pick door No. 2?'Is it to your advantage to switch?" Ask Marilyn! Or the BBC...

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X