At the AALS Annual Meeting today, the Section on Law and Social Sciences organized a panel entitled, “Designing Empirical Legal Inquiry.” The four panelists were Andrew Martin (Wash U), Lee Epstein (Northwestern), Tracey George (Vanderbilt), and Ahmed Taha (Wake Forest). The panel was designed for each of the participants to discuss different empirical methods to address the same question—How do judges make decisions?
Epstein kicked off the panel by discussing the Dark Side of Archival Research. While she normally reports about the rewards of archival treasure hunts, she wanted to make clear that while worthwhile, archival work requires money, includes uncomfortable workspace and lots of photocopies, and mandates the creation of detailed coding rules.
Martin discussed how cases filled words become lines of data filled with ones and zeros, often coding for votes. There a number of advantages to vote coding: easy to understand who wins/loses, data availability and reliability. Disadvantages include lack of available data on the lower courts, concerns of case selection bias, and possible validity concerns. In the future he expects case coding to be far richer with more focus on the lower courts, citation networks, and the actual text of the opinions.
Taha discussed his paper on How Panels Affect Judges: Evidence from U.S. District Courts. The data include the nearly 300 district court opinions on the constitutionality of the sentencing guidelines. What is interesting is that 14 district courts heard decisions in en banc. (Raise your hand if you knew district courts could do this!) Taha looked at whether panels created unanimity and why, combining his quantitative work with interviews of district court judges.
George presented “An Experimental Approach to the Study of the Judging” (on behalf of Chris Guthrie and Jeff Rachlinski who were unable to attend). Asking the question of whether judges can avoid bias, an example taken from (I think) Guthrie, Rachlinski & Wistrich, "Inside the Judicial Mind," 86 Cornell Lew Review 777 (2001), she focused on the strengths and limitations of between-group experimental methods that have a control and experimental group. The strengths: high internal validity, straight-forward statistics, can assess a particular independent variable because of random judge assignment. The limitations: external validity concerns, tells us about central tendencies, but not individuals, the need to go through IRB Human Subjects protocol, and the difficulty in recruiting the subject pool (especially judges).
The panel answered questions about selection bias in qualitative research, the difference between reliability versus validity, publishing empirical work in law reviews, and how can empirical results be conveyed in an accessible way to the reader.
In January, I posted a question about analysis of how the district judge transforms the record into a decision.
I repeat that question, which is tgaking on severe practicality, at least in the district court in one state (Mississippi).
The Democratic leadership, heavily influenced by its black constituency, had sought a decision to require party registration in order to preclude others from invating their primary. They goit that, but they also got a court order, which no one had requested, that photo ID be introduced, which is something with severe implications. One does not have to challenge whether the judge was right. The question is how one analyzes, i. e. how scholars are taught to analyze, the record and how the judge gets from gthe record to a decision.
Posted by: Matthew Holden, Jr. | 18 July 2007 at 09:38 PM
In this enterprise, is there a discussion, and possibly a citation to some recent scholarly literature, that shows how to describe and evaluate (a) what the Federal district court judge does in the course of proceeding and (b) what the judge does in transforming the record into his or her decision?
Posted by: Matthew Holden, Jr. | 26 January 2007 at 08:00 PM
Thanks Jason. Any chance you'd be willing to update this posting with links to all the papers?
Posted by: Ken | 12 January 2007 at 03:32 PM