Judgment aggregation

a bibliography on the discursive dilemma, doctrinal paradox and decisions on multiple propositions

sitemap 

overview ...

• introduction

• what is the "discursive dilemma" or "doctrinal paradox"?

• the premise- and conclusion-based procedures

• an impossibility theorem and more general developments 

papers on ...

• the doctrinal paradox and legal discussions

• the discursive dilemma

• general social-choice-theoretic models of judgment aggregation

  • parallels with Arrowian social choice (procedural considerations)

  • parallels with the Condorcet jury theorem (epistemic and probabilistic considerations)

• related work

** important disclaimer **

introduction

A new problem of social choice has attracted the attention of scholars in law, economics, political science, philosophy and computer science. How can a group of individuals aggregate the group members' individual judgments on some interconnected propositions into corresponding collective judgments on these propositions? Such aggregation problems occur in many different collective decision-making bodies, for example in committees, legislatures, judiciaries and expert panels.

Judgment aggregation is distinct from the more familiar problem of preference aggregation. But just as preference aggregation is illustrated by a paradox (Condorcet's paradox of cyclical majority preferences), so judgment aggregation is also illustrated by a paradox: the "discursive dilemma" or "doctrinal paradox".

This page provides a bibliography of online and published research on this paradox and on judgment aggregation more generally. 

what is the "discursive dilemma" or "doctrinal paradox"?

The "doctrinal paradox" illustrates the aggregation problem (Kornhauser and Sager 1986, 1993; Kornhauser 1992, the apparent first occurrence of the label "doctrinal paradox"; Chapman 1998). (Important earlier precursors in different frameworks are Guilbaud 1966, Wilson 1975, and Rubinstein and Fishburn 1986.)

Suppose that a three-member court has to make a judgment on whether a defendant is liable for a breach of contract. According to legal doctrine, the defendant is liable (proposition R) if and only if the defendant did some action X (proposition P) and the defendant had a contractual obligation not to do action X (proposition Q). Thus legal doctrine requires R<->(P&Q). Suppose that the individual judgments of the three judges are as in table 1.

Table 1. The "Doctrinal Paradox" or "Discursive Dilemma" (Conjunctive Version)

All three judges accept the rule R<->(P&Q). Further, judge 1 accepts both P and Q and, by implication, R. Judges 2 and 3 each accept only one of P or Q and, by implication, they both reject R. If the court applies majority voting on each proposition (including on R<->(P&Q)), it faces a paradoxical outcome. A majority accepts P, a majority accepts Q, a majority (unanimity) accepts R<->(P&Q), and yet a majority rejects R.

In earlier presentations of the problem under the name "doctrinal paradox", the logical connection rule R<->(P&Q) was not considered as a proposition on which the court explicitly makes a judgment by majority voting, but it was held fixed in the background as an exogenous constraint or "legal doctrine". This restriction was given up in more recent presentations of the problem under the name "discursive dilemma" (Pettit 2001b; List and Pettit 2002).

Propositionwise majority voting thus produces an inconsistent collective set of judgments, namely the set {P, Q, (R<->(P&Q)), not-R} (corresponding to the last row of table 1). This set is inconsistent in the standard sense of propositional logic: There exists no assignment of truth-values to propositions P, Q and R that makes all the propositions in the set simultaneously true. This outcome occurs although the sets of judgments of individual judges (corresponding to the first three rows of table 1) are all consistent. 

For a recent discussion of the paradox by Kornhauser and Sager, see Kornhauser and Sager (2004, cited below); for a response, see List and Pettit (2005). For a discussion of the discursive dilemma from a social epistemology perspective, see Goldman (2004) and List (2005). 

the premise- and conclusion-based procedures

Premise-based and conclusion-based procedures of decision-making have been proposed as possible escape-routes from the paradox. These procedures interpret propositions P and Q as premises, (R<->(P&Q)) as a rule of inference, and R as a conclusion.

According to the premise-based procedure, the group applies majority voting on each premise (i.e. propositions P and Q), but not on the conclusion (i.e. proposition R), and derives the collective judgment on that conclusion (i.e. R) on the basis of the appropriate logical connection rule (i.e. proposition R<->(P&Q)). Under this procedure, the group effectively ignores the majority verdict on the conclusion. In table 1, the premise-based procedure leads to the collective acceptance of the conclusion.

According to the conclusion-based procedure, the group applies majority voting only on the conclusion (i.e. R), but not on the premises (i.e. P and Q), ignoring the majority verdicts on them. In table 1, the conclusion-based procedure leads to the collective rejection of the conclusion.

Thus the premise-based and conclusion-based procedures may produce different outcomes.

For a discussion of the premise- and conclusion-based procedures, see Pettit (2001b) (a deliberative democracy and republican perspective), Bovens and Rabinowicz (2003, 2004) and List (2005) (an epistemic perspective, focusing on the truth-tracking capacities of the two procedures), Chapman (2002) (a common law perspective), Dietrich and List (2004a) (a discussion of the strategic incentives created by the two procedures).

an impossibility theorem and more general developments

It can be shown that the paradox is not just an artefact of majority voting, but that it illustrates a more general impossibility theorem.

A (judgment) aggregation procedure is a function which takes as its input a profile of individual sets of judgments across the members of a group, and which produces as its output a collective set of judgments. 

The sets of judgments of each individual are assumed to satisfy certain consistency conditions (completeness, consistency and deductive closure). 

For the following first impossibility result, the agenda of propositions on which judgments are to be made is assumed to contain  at least two "atomic" propositions (e.g. P, Q), one suitable "non-atomic" proposition (e.g. (P&Q)) and the negations of all these propositions.

Consider three simple conditions on an aggregation procedure (informally stated):

Universal Domain. An aggregation procedure accepts as admissible input any logically possible profile of individual sets of judgments.

Anonymity. All individuals have equal weight in determining the collective set of judgments.

Systematicity. The collective judgment on each proposition depends only on individual judgments on that proposition and the same pattern of dependence holds for all propositions.

Theorem (List and Pettit 2002). There exists no aggregation procedure (generating complete, consistent and deductively closed collective sets of judgments) which satisfies universal domain, anonymity and systematicity.

For extensions, generalizations and further impossibility results, see Pauly and van Hees (2003); Dietrich (2006); Gärdenfors (2006); Nehring and Puppe (2005a); Dietrich and List (2005b), (2005c), (2006); Dokow and Holzman (2005); Mongin (2006); Nehring (2005), (2006a).  

• All of these papers consider independence conditions weaker than systematicity (dropping the second part of systematicity, which requires the same pattern of dependence to hold for all propositions), but typically require agendas of propositions with richer logical interconnections. For a critique of the systematicity condition, see Chapman (2002).

• Among other results, Pauly and van Hees show that the anonymity condition of the theorem can be relaxed to a non-dictatorship condition; their paper also includes the first impossibility theorem in the judgment aggregation literature in which systematicity is weakened to independence.

• Nehring and Puppe (2005a) prove several results on judgment aggregation drawing on related results on the aggregation of preferences over vectors of properties in property spaces; for a key characterization result, see Nehring and Puppe (2005b); a Paretian-rationalist problem is presented in Nehring (2005); a result on oligarchic aggregation rules in Nehring (2006a).

• Dietrich and List (2005b) prove Arrow's theorem on preference aggregation as a corollary of an impossibility theorem on judgment aggregation; for an earlier comparison with impossibility results on preference aggregation, see List and Pettit (2001/2004). In both papers, preference orderings are represented as sets of binary ranking propositions in predicate logic. Nehring (2003) derives an Arrow-like theorem as a corollary of a theorem in the property-space framework.

• Dokow and Holzman (2005) identify an algebraic condition on the structure of logical interconnections between propositions that is necessary and sufficient for an Arrow-style impossibility result; their paper also develops connections between the judgment aggregation framework and Wilson's (1975) as well as Rubinstein and Fishburn's (1986) frameworks.

• Mongin (2006) provides an impossibility result based on an independence condition restricted to atomic propositions.

• Dietrich (2006) introduces a family of weakened independence conditions capturing the idea that some (but not all) propositions are relevant to certain others and proves several possibility and impossibility results under these conditions.. 

Other developments include the following.

• Pauly and van Hees (2003) and van Hees (2004) prove results on the aggregation of judgments expressed in many-valued logics and extend some impossibility results to this setting.

• Dietrich and List (2004a) analyse strategic voting and strategy-proofness in judgment aggregation. The analysis is based on a non-preference-theoretic notion of (non-)manipulability, but this notion is also related to a preference-theoretic notion of strategy-proofness similar to the one in standard social-choice-theoretic work on the Gibbard-Satterthwaite theorem.

• Domain restrictions sufficient for avoiding majority inconsistencies in judgment aggregation are discussed in List (2003) (see also correction) and Dietrich and List (2007).

• For parallels with the computer science literature on merging knowledge bases, see Pigozzi (2005a), (2005b), and for parallels with distance-based social choice, see Eckert and Pigozzi (2005).

• For impossibility results on the assignment of (expert or liberal) rights to group members, see Dietrich and List (2004b). 

• For a model of judgment aggregation in general logics (where propositions can be represented in several different logics, including propositional, predicate, modal and conditional logics), see Dietrich (2004).

• Quota rules (generalizing propositionwise majority voting) are  discussed in Dietrich and List (2005a).

• Pauly (2005) analyses the problem of axiomatizing judgment aggregation procedures subject to the constraint that axioms must be expressed in a minimal logical language.

• Gärdenfors (2006) weakens the rationality conditions on collective judgments from full rationality to deductive closure and consistency. For extensions of these results, see Dietrich and List (2006).

papers on ...

the doctrinal paradox and legal discussions

Bonnefon, Jean-François (2007) "How do Individuals Solve the Doctrinal Paradox in Collective Decisions? An Empirical Investigation," Psychological Science, forthcoming

Chapman, B. (1998) "More Easily Done than Said: Rules, Reason and Rational Social Choice," Oxford Journal of Legal Studies 18, pp. 293-329

Chapman, B. (2002) "Rational Choice and Categorical Reason," Pennsylvania Law Review, forthcoming

Ferejohn, J. (2001) "Statutes, Plans, Intentions: A Planning Theory of Legislation" (PDF) (source: http://www.law.nyu.edu/clppt/program2001/readings/index.html)

Kornhauser, L. A., and L. G. Sager (1986) "Unpacking the Court," Yale Law Journal 96: 82-117

Kornhauser, L. A. (1992) "Modelling Collegial Courts. II. Legal Doctrine," Journal of Law, Economics and Organization 8: 441-470

Kornhauser L. A., and L. G. Sager (1993) "The One and the Many: Adjudication in Collegial Courts," California Law Review 91: 1-51

Kornhauser, L. A., and L. G. Sager (2004) "Group Choice in Paradoxical Cases," Philosophy and Public Affairs 32: 249-76

List, C., and P. Pettit (2005) "On the Many as One," Philosophy and Public Affairs, 33(4): 377-390 (PDF) 

Nash, J. R. (2003) "A Context-Sensitive Voting Protocol Paradigm For Multimember Courts," Stanford Law Review 56: 75-159

the discursive dilemma

Baurmann, M., and G. Brennan (2005) "Majoritarian Inconsistency, Arrow Impossibility and the Comparative Interpretation: A Context-Based View," paper presented at the 2005 Public Choice conference (PDF) (source: http://www.pubchoicesoc.org/papers2005.html) 

Brennan G. (2001) "Collective Coherence?" International Review of Law and Economics 21(2): 197-211

Chapman, B. (2001) "Public Reason, Social Choice, and Cooperation," paper presented at the Eighth Conference on Theoretical Aspects of Rationality and Knowledge, University of Siena, held at Certosa di Pontignano, Italy, July 2001 (PDF) (source: http://chass.utoronto.ca/clea/confpapers.htm)

Chapman, B. (2002) "Rational Aggregation," Politics, Philosophy and Economics 1(3) 

Fallis, D. (2005) "Epistemic Value Theory and Judgment Aggregation," Episteme 2(1): 39-55

Goldman, A. (2004) "Group Knowledge Versus Group Rationality: Two Approaches to Social Epistemology," Episteme 1(1): 11-22 (PDF)

List, C. (2001) "Two Concepts of Agreement," The Good Society 11(1): 72-79 (revised follow-up paper as PDF)

List, C. (2004) "The Discursive Dilemma and Public Reason," Ethics, forthcoming (PDF)

List, C. (2005) "Group knowledge and group rationality: a judgment aggregation perspective," Episteme 2(1): 25-38 (PDF)  

List, C., and P. Pettit (2005) "Group Agency and Supervenience" (PDF)

Pettit, P. (2001a) "Akrasia, Collective and Individual" (PDF) (source: http://socpol.anu.edu.au/working.php3)

Pettit, P. (2001b) "Deliberative Democracy and the Discursive Dilemma," Philosophical Issues (supplement to Nous) 11: 268-99 (PDF) (source: http://socpol.anu.edu.au/working.php3)

Pettit, P. (2001c) "Groups with Minds of their Own" (PDF) (source: http://socpol.anu.edu.au/working.php3)

Peritz, D. (2003) "The Discursive Dilemma Dissolved" 

general social-choice-theoretic models of judgment aggregation 

parallels with Arrowian social choice (procedural considerations)

Cariani, F., M. Pauly and J. Snyder (2006) "Decision Framing in Judgment Aggregation" (PDF)

Claussen, C. A., and Ø. Røisland (2005) "Collective Economic Decisions and the Discursive Paradox", Norges Bank Working paper (PDF)

Dietrich, F. (2004) "A generalised model of judgment aggregation," Social Choice and Welfare, forthcoming (PDF)

Dietrich, F. (2005) "The possibility of judgment aggregation on agendas with subjunctive implications" (PDF)

Dietrich, F. (2006) "Judgment aggregation: (im)possibility theorems," Journal of Economic Theory 126(1): 286-298 (PDF) 

Dietrich, F. (2006) "Aggregation theory and the relevance of some issues to others" (PDF)

Dietrich, F., and C. List (2004a) "Strategy-Proof Judgment Aggregation" (PDF)

Dietrich, F., and C. List (2004b) "A Liberal Paradox for Judgment Aggregation," Social Choice and Welfare, forthcoming (PDF)

Dietrich, F., and C. List (2005a) "Judgment aggregation by quota rules," Journal of Theoretical Politics, forthcoming (PDF)

Dietrich, F., and C. List (2005b) "Arrow's theorem in judgment aggregation," Social Choice and Welfare, forthcoming (PDF)

Dietrich, F., and C. List (2005c) "The impossibility of unbiased judgment aggregation" (PDF)

Dietrich, F., and C. List (2006a) "Judgment aggregation on restricted domains" (PDF)

Dietrich, F., and C. List (2006b) "Judgment aggregation without full rationality," Social Choice and Welfare, forthcoming (PDF)

Dietrich, F., and C. List (2007) "Judgment aggregation with consistency alone" (PDF)

Dietrich, F., and C. List (2007) "Judgment aggregation under constraints," in Economics, Rational Choice and Normative Philosophy, T. Boylan and R. Gekker (eds.), London (Routledge), forthcoming (PDF)

Dietrich, F., and C. List (2007) "Majority voting on restricted domains" (PDF)

Dietrich, F., and P. Mongin (2007) "The Premiss-Based Approach to Judgment Aggregation" (PDF)

Dokow, E., and R. Holzman (2005) "Aggregation of Binary Evaluations" (PDF)

Eckert, D., and G. Pigozzi (2005) "Belief merging, judgment aggregation and some links with social choice theory," in Belief Change in Rational Agents, J. Delgrande et al. (eds.), Dagstuhl Seminar Proceedings 05321, IBFI, Schloss Dagstuhl, Germany, 2005 (PDF)

GarciaBermejo, Juan C. (2006) "Aggregating Judgments by the Majority Method" (PDF)

Gärdenfors, P. (2006) "An Arrow-like theorem for voting with logical consequences," Economics and Philosophy 22(2): 181-190 (PDF)

van Hees, M. (2004) "The Limits of Epistemic Democracy," Social Choice and Welfare, forthcoming (PDF)

Levi, I. (2004) "List and Pettit," Synthese 140(1-2): 237 - 242 (Link)    

List, C. (2003) "A Possibility Theorem on Decisions over Multiple Propositions," Mathematical Social Sciences 45(1): 1-13 (PDF) (correction)

List, C. (2004) "A Model of Path-Dependence in Decisions over Multiple Propositions," American Political Science Review 98(3): 495-513 (PDF)

List, C. (2006) "Which worlds are possible? A judgment aggregation problem" (PDF)

List, C., and P. Pettit (2002) "Aggregating Sets of Judgments: An Impossibility Result," Economics and Philosophy 18: 89-110 (PDF) 

List, C., and P. Pettit (2004) "Aggregating Sets of Judgments: Two Impossibility Results Compared," Synthese 140(1-2): 207-235; Australian National University Working Paper in Social and Political Theory W20, 2001 (PDF) 

Mongin, Philippe (2006) "Factoring Out the Impossibility of Logical Aggregation" (PDF)  

Nehring, K. (2003) "Arrow's theorem as a corollary," Economics Letters 80: 379-382 (PDF)

Nehring, K. (2005) "The Impossibility of a Paretian Rational" (PDF)

Nehring, K. and C. Puppe (2005a) "Consistent Judgement Aggregation: A Characterization" (PDF) (source: http://www.wior.uni-karlsruhe.de/LS_Puppe/Personal/Papers-Puppe/puppe_html)

Nehring, K. and C. Puppe (2007) "Justifiable Group Choice" (PDF)

Pauly, M. (2005) "Axiomatising Judgement Aggregation Procedures in a Minmial Logical Language" (PDF)

Pauly, M. and M. van Hees (2003) "Logical Constraints on Judgment Aggregation," Journal of Philosophical Logic, forthcoming (PDF)

Pigozzi, G. (2005a) "Collective Decision-Making without Paradoxes: A Fusion Approach," Synthese, forthcoming (PDF)

Pigozzi, G. (2005b) "Should we send him to prison? Paradoxes of aggregation and belief merging," in We Will Show Them: Essays in Honour of Dov Gabbay, Vol. 2, S. Artemov et al. (eds.), College Publications: 529-542 (PDF)

Zamora Bonilla, J. (2005) "Optimal Judgement Aggregation" (PDF)

parallels with the Condorcet jury theorem (epistemic and probabilistic considerations)

Bovens, L., and W. Rabinowicz (2003) "Democracy and Argument - Tracking Truth in Complex Social Decisions," in A. van Aaken, C. List and C. Luetge (eds.), Deliberation and Decision, Aldershot (Ashgate Publishing) (different version as PDF) (source http://pwp.netcabo.pt/0154943702/democracy.pdf) 

Bovens, L., and W. Rabinowicz (2004) "Democratic Answers to Complex Questions - an Epistemic Perspective," Synthese, forthcoming

List, C. (2005) "The Probability of Inconsistencies in Complex Collective Decisions," Social Choice and Welfare 24(1): 3-32 (PDF)

related work

Anscombe, G. E. M. (1976) "On Frustration of the Majority by Fulfillment of the Majority’s Will," Analysis 36(4): 161-168

Blackburn, S. (2001) "Group Minds and Expressive Harm", Maryland Law Review 60: 467-491

Brams, S. J., D. M. Kilgour and W. S. Zwicker (1997) "Voting on Referenda: the Separability Problem and Possible Solutions," Electoral Studies 16(3): 359-377

Brams, S.J., D. M. Kilgour and W. S. Zwicker (1998) "The paradox of multiple elections," Social Choice and Welfare 15: 211-236

Dietrich, F. (2004) "Opinion pooling under asymmetric information" (PDF)

Dietrich, F. and C. List (2007) "Opinion pooling on general agendas" (PDF) (an appendix with additional results)

Grofman, Bernard (1985) "Research Note: The Accuracy of Group Majorities for Disjunctive and Conjunctive Decision Tasks", Organizational Behavior and Human Decision Processes 35: 119-123

Guilbaud, G. Th. (1966) "Theories of the General Interest, and the Logical Problem of Aggregation," in P. F. Lazarsfeld and N. W. Henry (eds.), Readings in Mathematical Social Science, Cambridge/MA (MIT Press): 262-307

Hillinger, C. (1971) "Voting on issues and on platforms," Behavioral Science 16(6): 564-566

Kelly, J. S. (1989) "The Ostrogorski Paradox," Social Choice and Welfare 6: 71–76

Levmore, S. (2001) "Conjunction and Aggregation," Michigan Law Review 99 (Word) (source: http://www.law.uchicago.edu/faculty/levmore/publications.html)

Mazurkiewicz, M., and J. W. Mercik (2002) "Paradox of multiple elections - The probabilistic approach" (PDF) (source: http://polis.unipmn.it/epcs/papers/mercik.pdf)

Nehring, K., and C. Puppe (2005b) "On the Possibility of Strategy-Proof Social Choice: Non-Dictatorship, Anonymity and Neutrality" (PDF)  

Rubinstein, A., and P. Fishburn (1986) "Algebraic Aggregation Theory," Journal of Economic Theory 38: 63-77

Wilson, R. (1975) "On the Theory of Aggregation," Journal of Economic Theory 10: 89-99 

disclaimer

The owner of this page accepts no responsibility for the content (including the copyright) of any of the websites (including the online papers) to which it provides links. All the links on this page are to publicly accessible websites (typically in academic institutions), and it is assumed that the owners of those pages ensure that their material does not violate any copyright requirements.

compiled by Christian List