Empirical Legal Studies

The Costa Concordia Disaster & ELS

Though I posted on this paper last year, the unfolding tragedy involving the Costa Concordia reminded me anew of the paper's spectacular research design and execution. To refresh, in in Behavior Under Extreme Conditions: The Titanic Disaster, the authors exploit archival data and a quasi-natural experiment provided by the Titanic and Lusitania's tragic sinkings to test theories about how people behave in extreme situations.

The Titanic carried only 20 lifeboats, which could accommodate about half the people aboard, and deck officers exacerbated the shortage by launching lifeboats that were partially empty. It took the Titanic over 3 hours to sink, and failure to secure a seat in a lifeboat virtually guaranteed death. In contrast, the sinking of the Lusitania in 1915 took only 18 minutes from when the torpedo hit the ship. The paper compares individual-level data on the passengers and crew on from both ships which sheds helpful light on some specific questions: "Did physical strength (being male and in prime age) or social status (being a first- or second-class passenger) raise the survival chance? Was it favorable for survival to travel alone or in company? Does one's role or function (being a crew member or a passenger) affect the probability of survival? Do social norms, such as "Women and children first!" have any effect? Does nationality affect the chance of survival?"

Bankruptcy & Race

Although a few days old, anytime ELS scholarship lands in the NYT it likely warrants a post. The front page (above-the-fold) of last Saturday's NYT included an extensive summary of a forthcoming JELS article by Jean Braucher (Ariz.), Dov Cohen (Ill.--psych.), &  Bob Lawless (Ill.), Race, Attorney Influence, and Bankruptcy Chapter Choice. As the news story (and article) make clear, two related studies find that "Even after controlling for financial, demographic, and legal factors that might favor a chapter 13 filing, African Americans are much more likely to file chapter 13, as compared to debtors of other races."

The Visual Display of Quantitative Information (for ELS)

Andrew Gelman (Stats & Poli Sci; Columbia), who delivered a wonderful paper at the recent CELS conference, followed up with a recent post on graphical communication for empirical legal scholarship. As you will see, Andrew's post pivots on a thoughtful comment from Dan Kahan (Yale).

Getting Underneath Data and Results

Over the years I have repeatedly emphasized to my students in empirical methods classes of the need to "get underneath" the data and results, particularly for secondary analyses. By that I mean researchers invariably benefit from on-the-ground insights into and outside perspectives on what their data (and research design) actually capture and results suggest. Kyle Graham's helpful post over at Concurring Opinion illustrates how this general point can work when evaluating a possible empirical project.

Influence of Mandatory and Discretionary Court Dockets on Case Outcomes

In an intriguing paper forthcoming in the Journal of Empirical Legal Studies (2012), Does the Judge Matter? Exploiting Random Assignment on a Court of Last Resort to Assess Judge and Case Selection Effects, Ted Eisenberg (Cornell), Talia Fisher (Tel Aviv), and Issachar Rosen-Zvi (Tel Aviv), exploit a database of criminal cases appealed to the Israel Supreme Court. In the course of seeking to assess individual judge effects on case outcomes, and controlling for the usual complexities that contribute to non-random case assignment to justices, the authors lever a factor that distinguishes the Supreme Courts in Israel and the U.S. Specifically, the Israel Supreme Court's docket includes an important cohort of mandatory cases. The paper notes how "studies limited to discretionary jurisdiction case outcomes can distort perceptions of judges’ preferences." The abstract follows.

"We study 1410 criminal law cases appealed to the Israel Supreme Court in 2006 and 2007 to assess influences on case outcomes. A methodological innovation is accounting for factors — case specialization, seniority, and workload — that modify random case assignment. To the extent one accounts for nonrandom assignment, one can infer that case outcome differences are judge effects. Individual justices cast 3986 votes and differed by as much as 15 percent in the probability of casting a vote favoring defendants. Female justices were about two to three percent more likely than male justices to vote for defendants but this effect is sensitive to including one justice. Defendant gender was associated with outcome, with female defendants about 17 percent more likely than male defendants to receive a favorable vote on appeal. Our data’s full samples of mandatory and discretionary jurisdiction cases allow us to show that studies limited to discretionary jurisdiction case outcomes can distort perceptions of judges’ preferences. Justices’ ordinal rank in rate of voting for defendants or the State was uncorrelated across mandatory and discretionary jurisdiction cases. For example, the justice who sat on the most criminal cases was the fourth (of 16 justices) most favorable to the State in mandatory jurisdiction cases but the twelfth most favorable in discretionary jurisdiction cases. This result casts doubt on studies of judges on discretionary jurisdiction courts, such as the U.S. Supreme Court, in which votes in the selection process are not observed."

Criminal Sentencing, Booker, and IP Cases

Owing to external shocks, such as the Booker decision, U.S. Sentencing Guidelines panel data provide a number of research opportunities. In an accessible paper, Post-Booker Judicial Discretion and Sentencing Trends in Criminal Intellectual Property Cases: Empirical Analysis and Societal Implications, Aaron Rabinowitz (Temple) assesses Booker's influence on sentencing for IP convictions. A summary of the paper's findings, from an excerpted abstract, follow.

"... [T]hat sentences imposed on intellectual property offenders deviate from the advisory Guidelines in two out of every three cases; prosecutors seek and judges reduce sentences for intellectual property crimes more frequently than for other comparable crimes; and judge-initiated deviations occur after Booker about twice as frequently for intellectual property offenders than for other offenders, whereas such judge-initiated deviations before Booker occurred less frequently than for crimes in general or for other economic crimes."

The Journal of Law and Courts

As Chair of the Law and Courts section of the American Political Science Association, I am delighted to announce the inauguration of the Journal of Law and Courts, the official journal of the section.

The Journal is edited by David Klein, of the University of Virginia's Department of Politics, and published by the University of Chicago Press. The JLC is a double-blind peer-reviewed, single-submission journal, indexed by Lexis-Nexis, Westlaw, EBSCO, JSTOR, and others. While formally an APSA section journal, the JLC aims to be the premier outlet for the publication of work on law, courts, and things judicial from a wide range of perspectives.  More information about the journal and instructions for authors can be found here.

Submissions are being accepted now, via the journal's Editorial Manager page. We anticipate publication of the inaugural issue in spring 2013.

Efforts to Randomize & Efficacy of Representation

In a pair of related, though distinct, papers, Jim Greiner (Harvard Law), Cassandra Pattanayak (Harvard--Stats), and Jonathan Hennessy (Harvard--Stats) seek to exploit randomized selection in a challenging effort to explore the efficacy of legal representation. Findings from the two studies point in slightly different directions.

In Limits of Unbundled Legal Assistance: a Randomized Study In A Massachusetts District Court and Prospects for the Future and How Effective Are Limited Legal Assistance Programs? A Randomized Experiment in a Massachusetts Housing Court the authors present results from a studies that share a setting-- summary eviction in Massachusetts. Specifically, the authors "persuaded entities conducting two civil Gideon pilot programs to randomize which potential clients would receive offers of traditional attorney-client relationships from professional service provider staff attorneys and which would receive only limited (“unbundled”) assistance. In both pilot programs potential clients were defendants in housing eviction proceedings, and both programs were oversubscribed."

"In the District Court Study, almost all study-eligible eviction defendants received limited assistance in the form of help in filling out answer and discovery request forms, and most also attended an instructional session on the summary eviction process. After receiving this limited assistance, each member of a randomly selected treated group received an offer of a traditional attorney-client relationship from one of the legal services provider’s staff attorneys; each member of the corresponding randomly selected control group received no such offer.... At least for the clientele involved in this District Court Study, a clientele recruited and chosen by the service provider’s proactive, timely, specific, and selective outreach and intake system, an offer of full representation mattered."

"In the Housing Court Study, all study-eligible potential clients could (and most did) receive assistance in filling out answers and discovery requests. In addition, occupants not offered a traditional attorney-client relationship from the provider's staff attorneys, the control group, received a referral to that provider's lawyer for the day program. The lawyer for the day program provided same-day-only representation in hallway settlement negotiations and mediation sessions but not in court appearances or in filing motions.... We find no statistically significant evidence that the service provider's offer of full, as opposed to limited, representation had a large (or any) effect on the likelihood that the occupant would retain possession, on the financial consequences of the case, on judicial involvement in or attention to cases, or on any other litigation-related outcome of substantive import. To the contrary, treated and control group point estimates are close to one another."

The Influence of Race on eBay Auctions

Regardless of what one might think about the prospects for a "Post-Racial" society, one could plausibly assume that racial effects are less likely to emerge in an auction setting. Results from a clever study challenge that assumption, however. In Race Effects on Ebay, Ian Ayres (Yale), Mahzarin Banaji (Harvard--Psych), and Christine Jolls (Yale) leverage eBay, manipulate race, and assess racial effects on the price of baseball cards. The abstract follows.

"We investigate the impact of seller race in a field experiment involving baseball card auctions on eBay. Photographs showed the cards held by either a dark-skinned/African-American hand or a light-skinned/Caucasian hand. Cards held by African-American sellers sold for approximately 20% ($0.90) less than cards held by Caucasian sellers, and the race effect was more pronounced in sales of minority player cards. Our evidence of race differentials is important because the on-line environment is well controlled (with the absence of confounding tester effects) and because the results show that race effects can persist in a thick real-world market such as eBay."

Statistical v. Legal "Significance": Matrixx-Inspired Scholarship

In Matrixx Initiatives Inc. v. Siracusano, the Supreme Court considered “[w]hether a plaintiff can state a claim under § 10(b) of the Securities Exchange Act and SEC Rule 10b-5 based on a pharmaceutical company’s nondisclosure of adverse event reports even though the reports are not alleged to be statistically significant.”

The Court said "yes," and argued that a reasonable investor might want to know of such reports if they are sufficiently extensive and disturbing that they could prompt the FDA to take some action or might lead to costly lawsuits. Thus, even non-statistically significant information can be material and enough to trigger a legal duty to disclose.

In Trapped in the Matrixx: The U.S. Supreme Court and the Need for Statistical Significance, David Kaye (Penn St.) defends the Court's interpretation of the pleading rule but argues for a narrow reading of dicta in the opinion about proof of causation in product liability cases. Kaye's paper "suggests that the unanimous opinion conflates issues of study design with statistical significance. Disentangling these concepts shows that the Court’s remarks do not address the limited value of adverse event reports in establishing causation in toxic tort litigation."