For experimental judicial decisionmaking studies, the "gold-standard" research design includes using actual judges as participants. My Cornell colleague (and good friend) Jeff Rachlinski is a leading scholar in this space. For an array of reasons (including practical reasons), too much of the experimental research in this field is hampered by researchers' use of students (including law students) as the experimental participants. To the extant that researchers seek to generalize results from judicial decisionmaking studies to the "universe of judges," conventional wisdom suggests that the use of students in these experiments raises important external validity questions.
Of course, such (and most) "conventional wisdoms" rest on empirical assumptions. And in a recent paper, Can Law Students Replace Judges in Experiments of Judicial-Making?, Holger Spamann (Harvard) and Lars Klohn (Humboldt) submit this question to an experiment. The answer to the question they pose in their paper's title?: "unfortunately, no." The paper's abstract follows.
"Experimental research on judicial decision-making is hampered by the difficulty of recruiting judges as experimental participants. Can students be used in judges’ stead? Unfortunately, no. We ran the same high-context 2×2 factorial experiment of judicial decision-making focused on legal reasoning with 31 U.S. federal judges and 91 elite U.S. law students. We obtained diametrically opposed results. Judges’ decisions were strongly associated with one factor (sympathy, i.e., bias) but not the other (law). For students, it was the other way around. Equality between the two groups is strongly rejected. Equality of document-view patterns—a proxy for thought processes—and written reasons is also strongly rejected."
Comments