Empirical Legal Studies

How to Write an Empirical Research Paper

For any students (or faculty, for that matter) contemplating a summer research project, Andrew Gelman (Columbia--Poli Sci) offers quite helpful, quick advice on how to approach writing up a research paper (here).

Call for Papers: AI and the Law

Robert Richards, proprietor of the Legal Informantics Blog, points out a call for papers for a special issue of Artificial Intelligence and Law focused on "modeling policy making."  Of particular note for ELS folks is that the special issue welcomes submissions on "the first three phases of the policy cycle: agenda setting, policy analysis, and lawmaking."

A History of "p" Levels

While most ELS Blog readers understand traditional significance ("p") levels, few understand how (or why) "0.05" emerged as the standard for statistical significance. In The Adoption of Significance Tests by the Scientific Community: An Empirical Analysis, David A. Gully (Columbia--Engineering) discusses the adoption of the 0.05 standard. A excerpted abstract follows:

"This paper adds to the literature by determining the timing and level of acceptance of common tests of statistical inference. Using the archives of the Royal Society, we examined 574 research studies published between 1926 and 1997, by which point adoption was virtually complete... We detect the presence of several influences on the rate of adoption, which may include prior custom, the nature of empirical research topics being reported, the increasing ease of computer processing, and possibly journal editorial policies. We find that confidence/significance testing has been adopted by a majority of the scientific community for over 50 years; the customary reliance on 95 percent confidence (five percent significance) is upheld by the data; and that confidence intervals and critical significance levels are both widely reported and often together in recent decades."

The ELS "Sandbox" & Boundary Line-Drawing

Brian Leiter recently triggered a mild dust-up (here and here) when he sought to identify loose boundaries for construing legal scholars as "empirical." Lior Strahilevitz (Chicago) exploits the frustration Leiter uncovers and re-directs it into a discussion over at PrawfsBlawg that "might be helpful to people who are skeptical about empiricist hiring in general on law faculties." Specifically, Lior thoughtfully opines about "how to hire entry-level quantitative empiricists with PhDs in disciplines like Political Science or Economics, as well as a coda about what many empiricists should be doing as the 'field' matures." Well worth a read.

The ACLU's Effectiveness as Amicus Curiae

One of my honors advisees this term has just completed her thesis, an empirical examination of the ACLU's ability to influence the U.S. Supreme Court through filing amicus curiae briefs during the Burger and Rehnquist Courts.  It's a careful study, combining qualitative and quantitative methods. The thesis has a number of interesting findings; I won't go into all the details, but I wanted to share this:

It's a plot of the odds ratio for the presence of an ACLU brief (from a series of justice-specific logistic regressions that also included a host of controls) where the outcome variable is a pro-ACLU vote.  The vertical line at 1.0 corresponds to "no marginal association;" values less than one are negative associations, while values above one are positive. (I'm omitting the CIs because they make the plot very messy.)

The interesting thing, of course, is that justices widely perceived as more politically liberal also tend to be those that are most "influenced" by the ACLU's amicus briefs.

Also, to all those NYC firms out there: My advisee, Shannon Azzaro, will be starting law school at Fordham in the fall, with a focus on intellectual property.  I understand that her 2013 summer is still free...

Bias in ABA Judicial Performance Evaluations

UNLV's Rebecca Wood just posted an interesting (and potentially explosive) paper, "Judicial Performance Evaluations as Biased and Invalid Measures: Why the ABA Guidelines are Not Good Enough," on SSRN. The abstract:

Judicial performance evaluations (JPEs) are an important part of the judicial selection process in the states, particularly those using a version of the merit plan. All states that use JPEs follow the ABA’s Guidelines (1985), which claim to minimize the potential for unconscious bias through the use of behavior-based evaluation. But these measures have yet to be subjected to rigorous analysis. This analysis of the “Judging the Judges” survey of Nevada attorneys provides such an analysis. After controlling for objective measures of judicial performance, gender and race still contribute significantly to the scores on all of the behavior-based measures implemented in the Nevada poll. I find evidence of significant unconscious bias, as social cognition theory would predict. The analysis also cast serious doubt on the overall validity of these measures of judicial quality. This result raises serious questions about the validity and fairness of JPEs around the country.

 

 

Perils of Self-Reported Data

I often caution students contemplating using self-reported data and urge them to consider observational data where possible (and appropriate to the research question and design). Why Bother Asking? The Limited Value of Self-Reported Vote Intention, by Rogers, Todd Rogers (Harvard--Kennedy School) and Masa Aida (Greenberg Quinlan Rosner), contribute empirical support to my intuition, certainly as it relates to predicted voting behavior. The findings' implications for an array of sub-fields are important. An excerpted abstract follows.

"Phone surveys from three elections, including one survey experiment, are analyzed to compare respondents’ pre-election vote intention with their actual voting behavior using administrative records (N=29,403). Unsurprisingly, many who predict that they will vote actually do not vote. More surprisingly, many who predict that they will not vote actually do vote (29% to 56%). Records of past voting behavior predicts turnout substantially better than self-prediction. Self-prediction inaccuracy is not caused by lack of cognitive salience of past voting, or by inability to recall past voting. Moreover, self-reported recall of turnout in one past election predicts future turnout just as well as self-prediction. We discuss implications for political science research, behavioral prediction, election administration policy, and public opinion."

Death Penalty Data in Delaware

My Cornell colleagues--Sheri Johnson, John Blume, Ted Eisenberg, Valerie Hans, and Marty Wells--report on the administration of the death penalty in The Delaware Death Penalty: An Empirical Study. The excerpted abstract, below, summarizes their findings.

"Our conclusions consist of three main observations. First, Delaware’s reversal rate in capital cases, 44 %, while substantial, is also substantially less than that of other jurisdictions. This may not be surprising, given Delaware’s emphasis for much of the time period on judge sentencing and that jury verdicts offer more opportunities for reversal. Indeed, reversal rates during the jury sentencing period approximate the national average. Second, judge sentencing in Delaware results in more death sentences, a result consistent with greater harshness being the motivation behind the statutory change to judge sentencing. This effect, is more pronounced in Delaware than in other states. Third, we find a dramatic disparity of death sentencing rates by race, one substantially more pronounced than in other jurisdictions. Race matters in capital sentencing, as David Baldus told us more than a quarter of century ago, and we need to continue to pursue knowledge about where, when, and how it matters."

ELS & International Law

While empirical work on international law issues has lagged compared empirical work in other areas, partly owing to a paucity of helpful data, change is afoot. In The Empirical Turn in International Legal Scholarship, Gregory Shaffer (Minn.) and Tom Ginsburg (Chicago), make the case, describe the recent "turn," and explore the emerging scholarly opportunities. The abstract follows:

"This article presents and assesses a new wave of empirical research on international law. Recent scholarship has moved away from theoretical debates over whether international law 'matters,' and focuses instead on exploring the conditions under which international law is created and produces effects. As this empirical research program has matured, it has allowed for new, mid-level theorizing that we call 'conditional international law theory.'"

American Bar Foundation (ABF) Call for Visiting Scholars

The American Bar Foundation (ABF)  invites scholars to join our intellectual community for the 2012-2013 academic year.

The ABF encourages national and international scholars on leave or sabbatical to take advantage of our diverse community and excellent facilities.  We offer an office, telephone, and computer, but no stipend.

Preference will be given to visitors whose scholarship coincides with the research agenda of the ABF and who will be in residence full-time for all or part of the year.  Summer visits are possible. Visitors are expected to participate in the intellectual life of the ABF, including a weekly seminar.

If you have an interest in this opportunity, please send an email to Robert Nelson at rnelson@abfn.org, subject line: Visiting Scholars Program, which states (1) the topic on which you are working, (2) the preferred dates for residence, (3) the days each week you would expect to be at the ABF, and (4) attach a CV.  Applications should be received by April 1, 2012.  Applications will be considered as space allows.  The ABF Appointments Committee will review applications and prospective visitors will be notified accordingly.