Empirical Legal Studies

Opportunity for Empirically-Minded Scholars: NSF-Funded Conference on Reconstituting Research in Access to Civil Justice

Jim Greiner (Harvard) brought the following opportunity to my attention and thought it might interest ElsBlog readers. Anyone with follow-up questions should direct them to Jim at: "jgreiner at law.harvard.edu".

"A group of researchers and field professionals in access to civil justice (A2J) in the United States is soliciting applications to attend a two-day Workshop to be held in Chicago, Illinois, on December 7-8, 2012. The application process is short, consisting of an update c.v. and a 300-word essay on a specific research question or area of inquiry regarding access to civil justice that the applicant thinks is understudied and would benefit from qualitative or quantitative empirical research. The Workshop opens with a poster session and town hall meeting on the afternoon of Friday, December 7, bringing together scholars and practitioners from many perspectives to identify and explore access to justice research needs. On the following day, Saturday, December 8, the Workshop will convene a smaller, closed session to push forward the work of revitalizing A2J research. The Workshop organizers have NSF funding to defray all reasonable expenses for travel, lodging, and food.

This Workshop presents an unusual opportunity for empirically minded researchers, particularly beginning researchers or those who have previously not focused on the administration and delivery of civil justice. A broad cross-section of field personnel, academics, funders, and government officials have been invited to the Workshop, and one goal is to create a network for future inquiries. Questions can be directed to Jim Greiner, jgreiner at law.harvard.edu. The application can be found here (scroll down the page a bit)" and is due by September 28, 2012.

Graphically Displaying Uncertainty

A nice pair of posts from Andrew Gelman's (Columbia--Poli Sci, Statistics) blog (here) illustrates and briefly discusses the virtues of visually conveying uncertainty in regression results. The initial post illustrates two competing visual approaches. A follow-up post includes (or links to) helpful coding.

Empirical Training Workshop at CELS

The Society for Empirical Legal Studies Executive Director, Dawn Chutkow, passed along the following that might interest those planning to attend the CELS at Stanford in November.

Ted Eisenberg (Cornell) will conduct an Empirical Training Workshop on November 8 - 9, in connection with CELS 2012 at Stanford Law School. Enrollment is limited. The workshop begins the day before the conference. A brief description and a link for more information follow.

The Empirical Training Workshop is intended for professors and students who seek an introduction to the statistical and programming skills needed to conduct quantitative empirical legal research. Professor Eisenberg will guide participants through an intensive 10-hour course on statistical analysis in the legal context. Pre-registration and a small fee are required.

More information on the Workshop is found here.

The Internationalization of ELS - Brazil

One of the more important and interesting emerging trends involves the strong--and growing--interest in all things ELS from scholars based outside of the U.S. Evidence of this growing internationalization of ELS abounds and can be found in submissions to and publications in the Journal of Empirical Legal Studies, participation in the annual SELS conference, and the increased methodological sophistication demonstrated in papers by non-U.S. scholars circulating on SSRN. What I have consistently heard from an array of international scholars, however, are laments about the comparative paucity of ELS-related institutions outside of the U.S.

Consequently, developments on this final point warrant attention. In Brazil, under the leadership of Marcelo Guedes Nunes, The Brazilian Jurimetrics Association ("ABJ") is undertaking the critical task of developing a scholarly infrastructure to support ELS-related work in Brazil.

"The Brazilian Jurimetrics Association is a nonprofit organization gathering researchers in Law and Mathematics, which aims to investigate and promote the use of statistics and probability in the study of Law and its institutions. The ABJ has three main missions. The first is to gather and encourage researchers to investigate and describe the decision-making processes in which individual and specific norms are created. The second is to discipline Jurimetrics as a branch of legal knowledge, defining its premises, its fundamentals, concepts, and key relationships. The third is to collaborate with public and private entities’ strategic efforts to improve the mechanisms for judicial interpretation of laws through the development of laws and court management."

Having participated in the ABJ's Second Annual Brazilian Jurimetrics Conference I can attest to the first-rate empirical work undertaken by a growing number of outstanding scholars in Brazil. Anyone interested in the development of ELS work in a "BRIC" nation should learn about the ABJ (for those on Twitter: @abjurimetria).

A Blast From the Past

Bob Lawless (Ill.) and Paul Stancil (Ill.) were kind enough to pass along the following quote:

"There has been a tendency on the one hand to deify and on the other to decry the results of tabulations of court business. It is easy to go to either extreme. More soundly, however, students of law administration are learning the proper function of such mass statistics in providing trustworthy facts, so far as they go, of court activities which may be used to verify, support, disprove, or suggest general hypotheses. The facts must be of such general nature as will lend themselves to average or mass verification, such as the nature of the general run of business in the courts, the character of the parties to the suit, the general methods of termination of cases, whether by court, jury or some other form of trial or by agreement or withdrawal and so on; but within these limits the facts may be definitely ascertained. Moreover, facts in themselves do not prove what should be the policy of law administration or the direction of reform therein. Their function is but to cast light upon the factors which should shape the rules of policy.”

What makes the quote particularly notable was that it comes from The American Law Institute, A Study of the Business of the Federal Courts (1934).

Maintenance Issues

Apologies for the recent service interruption owing to platform migration issues.

More on "More Guns, Less Crime"

In The Impact of Right to Carry Laws and the NRC Report: The Latest Lessons for the Empirical Evaluation of Law and Policy, John Donohue (Stanford), Abhay Aneja (Stanford), and Alexandria Zhang (Johns Hopkins), revisit and synthesize empirical evidence about the policy efficacy of controversial "right-to-carry" laws. Also notable is that the paper helpfully addresses important methodological concerns incident to panel data methods. Insofar as a contested history frames the empirical "right-to-carry" debate, some brief background is warranted, followed by an excerpted abstract.

In 2005, the National Research Council (NRC) evaluated the “More Guns, Less Crime” hypothesis using county-level crime data for the period 1977-2000. 17 of the 18 NRC panel members essentially concluded that the existing research was inconclusive on whether "right-to-carry" laws increased or decreased crime.

"We evaluate the NRC evidence, and improve and expand on the report’s county data analysis by analyzing an additional six years of county data as well as state panel data for the period 1977-2006. We also present evidence using both a more plausible version of the Lott and Mustard specification, as well as our own preferred specification (which, unlike the Lott and Mustard model used in the NRC report, does control for rates of incarceration and police). While we have considerable sympathy with the NRC’s majority view about the difficulty of drawing conclusions from simple panel data models, we disagree with the NRC report’s judgment that cluster adjustments to correct for serial correlation are not needed. Our randomization tests show that without such adjustments the Type 1 error soars to 44-75 percent. In addition, the conclusion of the dissenting panel member that RTC laws reduce murder has no statistical support.

Our paper highlights some important questions to consider when using panel data methods to resolve questions of law and policy effectiveness. Although we agree with the NRC’s cautious conclusion regarding the effects of RTC laws, we buttress this conclusion by showing how sensitive the estimated impact of RTC laws is to different data periods, the use of state versus county data, particular specifications, and the decision to control for state trends. Overall, the most consistent, albeit not uniform, finding to emerge from both the state and county panel data models conducted over the entire 1977-2006 period with and without state trends and using three different specifications is that aggravated assault rises when RTC laws are adopted."

Statistical Detection of Falsified Data

While I truly hope not a sign of the times, as the title implies, Just Post It: The Lesson from Two Cases of Fabricated Data Detected by Statistics Alone, illustrates how simple statistical tests can be used to detect falsified data. Interestingly, Uri Simonsohn's (Penn-Wharton) main point involves a plea for journals to make available raw data used to support findings in published papers. That such a policy may assist in rooting out fraudulent papers (and deterring such unethical conduct) is a positive spillover. The abstract follows.

"I argue that journals should require authors to post the raw data supporting their published results. I illustrate some of the benefits of doing so by describing two cases of fraud I identified exclusively through statistical analysis of reported means and standard deviations. Analyses of the raw data provided important confirmation of the initial suspicions, ruling out benign explanations (e.g., reporting errors; unusual distributions), identifying additional signs of fabrication, and also ruling out one of the suspected fraudster’s explanations for his anomalous results. If we want to reduce fraud, we need to require authors to post their raw data."

Data on Health Care Quality's Imperviousness to Tort Reform

A recent paper by a Cornell colleague, Mike Frakes, will interest tort scholars (as well as health law specialists and economists). In Does Medical Malpractice Deter? The Impact of Tort Reforms and Malpractice Standard Reforms on Healthcare Quality, Mike exploits data from National Hospital Discharge Surveys to assess the impact of various tort reforms on avoidable hospitalization rates. Frakes finds only "modest" evidence of deterrence. The abstract follows.

"Despite the fundamental role of deterrence in the theoretical justification for medical malpractice law, surprisingly little evidence has been put forth to date bearing on its existence and scope. Using data from the 1979 to 2005 National Hospital Discharge Surveys and drawing on an extensive set of variations in various tort measures (e.g., damage caps) and malpractice standard-of-care rules (Frakes 2012a), I estimate a small and statistically insignificant relationship between malpractice forces and two metrics of health care quality emphasized by the Agency for Healthcare Research and Quality: (1) avoidable hospitalization rates (reflective of outpatient quality) and (2) inpatient mortality rates for selected medical conditions. At most, the evidence implies an arguably modest degree of malpractice-induced deterrence. For instance, at one end of the 95% confidence interval, the lack of a non-economic damages cap (indicative of higher malpractice pressure) is associated with only a 4% decrease in avoidable hospitalizations."

"False" Confessions, Plea Bargains, Experimental Designs & IRB

A recent paper now circulating on SSRN, The Innocent Defendant’s Dilemma: An Innovative Empirical Study of Plea Bargaining’s Innocence Problem, intrigues for an array of reasons. The subject matter--defendants' willingness to plead to offenses they did not commit incident to a plea bargain--certainly provokes. Also notable is the research design employed. The researchers, Lucian Dervan (S. Ill.) and Vanessa Edkins (psychology, Florida Tech.), constructed an experiment in which college students actually felt they were "accused of cheating and presented with her own incentives to admit wrongdoing and save the university the time and expense of proceeding before a disciplinary review board." The study involved "dozens of college students, and taking place over several months ... revealed that plea bargaining’s innocence problem is not isolated to an obscure and rare set of cases." Finally, insofar as this experiment involved human subjects, that the researchers managed to get their research project through IRB struck me as something of a minor miracle. An excerpted abstract follows.

"Strikingly, the authors’ study found that over half of the participants were willing to falsely admit guilt in return for a perceived benefit. This finding not only brings finality to the long-standing debate regarding the possible extent of plea bargaining’s innocence problem, but also ignites a fundamental constitutional question regarding an institution the Supreme Court reluctantly approved of in 1970 in return for an assurance it would not be used to induce innocent defendants to falsely admit guilt."