Empirical Legal Studies

Empirical Research for Law & Policy: A Cautionary Tale Through Three Case Studies

A recently circulated paper by Richard Lempert (Michigan), Empirical Research for Public Policy: With Examples from Family Law and Advice on Securing Funding, will interest many.  In it Lempert explores practical challenges that confront many empirical legal scholars through three case studies.  The paper "discusses the use of empirical research for public policy and suggests a number of cautions that should be exercised in generalizing from empirical studies to public policy with a focus on the role of theory and both internal and external validity."

As Prof. Lempert notes in his paper: "There are, however, many dangers in relying on empirical work, rather than normative arguments, to influence policy. Empirical work is social science, and the standards of social science as well as what it determines can change. Methods, data availability and, to some extent, data quality are improving all the time. What was state of the art research yesterday may be perceived as hopelessly flawed tomorrow, and reanalyzing even the same data using more modern methods may reveal that the earlier advice was not just unsupported by the data but contrary to the policies that the data, properly analyzed, suggest."  Of course, Prof. Lempert goes on to note that, amid this uncertainty "policy makers must act" and, as well, judges must decide cases.

New York State Judge Sacked

By a 9 to 1 vote, the New York State Commission on Judicial Conduct recently removed Judge Robert Restaino from his position as a judge of the Niagara Falls City Court. According to the commission:

“In an egregious and unprecedented abuse of judicial power, respondent committed 46 defendants into police custody in a bizarre, unsuccessful effort to discover the owner of a ringing cell phone in the courtroom. In doing so, he inexplicably persisted in his conduct over some two hours, questioning the defendants individually about the phone before committing them into custody, and ignoring the pleas of numerous defendants who protested that his conduct was unfair and pleaded that he reconsider. Respondent’s conduct, which resulted in the unjustified detention of the defendants for several hours and the incarceration of 14 defendants in the County Jail, caused irreparable damage to public confidence in the fair and proper administration of justice in his court.”

The commission’s decision is dated November 13, 2007, but the press is only now reporting the story.  The commission’s decision, which includes both a concurrence and a dissent, is here.

New ICPSR Data Releases

New ICPSR data releases here.   One that caught my eye is the 2004 update for the National Judicial Reporting Program Series.  Although the 2004 sample is a cross-section, the series enables longitudinal analysis from 1986 to 2004.  Here is a description for the 2004 dataset:

This data collection provides detailed information on the sentences and characteristics of convicted felons based on data collected from state courts. The 2004 survey was based on a sample of 300 counties selected to be nationally representative. The collection contains sociodemographic information such as age, race, and sex of the felon. Types of offenses committed include homicide, rape, and robbery. Data can be analyzed at the national level or by individual counties.

There is a lot of county-level demographic data available from the Census Bureau, including shape files for GIS mapping software.   All of these data sources can be linked together in a matter of hours thanks to Census Bureau FIPS codes.   For a criminal law scholar interested in empirical research, this is great (and free) starter data set. 

More From BJS: Pretrial Release of Felony Defendants in State Courts

Pretrial Release of Felony Defendants in State Courts presents findings on the pretrial release phase of the criminal justice process using data collected from a representative sample of felony cases filed in the 75 largest U.S. counties in May during even-numbered years from 1990 to 2004. It includes trends on pretrial release rates and the types of release used. Pretrial release rates are compared by arrest offense, demographic characteristics, and criminal history. Characteristics of released and detained defendants are also presented. Rates of pretrial misconduct including failure to appear and rearrest are presented by type of release, demographic characteristics, and criminal history.  Findings from the report include:

• About 3 in 5 felony defendants in the 75 largest counties were released prior to the disposition of their case.
• Surety bond surpassed release on recognizance in 1998 as the most common type of pretrial release.
• Defendants on financial release were more likely to make all scheduled court appearances.

Launch of International Data Resource Center (IDRC)

The Inter-University Consortium on Social and Political Research (ICPSR) has just launched a new data initiative with an international focus.  From the ICPSR website:

ICPSR is pleased to announce the launch of the International Data Resource Center (IDRC) Web site. The IDRC serves as a central entry point into ICPSR's international data holdings. Dr. Denise DeGarmo, Associate Professor of Political Science at Southern Illinois University, Edwardsville, developd the site while in residence at ICPSR as an ICPSR Official Representative Sabbatical Fellow.

International data available through the IDRC includes conflict data, economic data, data on electoral systems and political behavior, environmental data, health data, data pertaining to the human dimension of international relations, public opinion data, and data on international organizations. Searches for international data can be conducted using several different methods, including subject searches, series data searches, or the IDRC's interactive map interface.

The IDRC website is very attractive and easy to navigate.  It also includes access to several foundational datasets. 

Methodological Fit

The most recent issue of the Academy of Management Review contains a must-read for the legal empiricists who lack formal social science training (e.g., me):  "Methodological Fit in Management Field Research." The article, by Amy Edmondson (Harvard Business School) and Stacey McManus (Monitor Executive Development), provides a remarkably accessible and constructive framework for research design, including the respective roles of qualitative and quantitative methods.  Here is the abstract:

Methodological fit, an implicitly valued attribute of high-quality field research inorganizations, has received little attention in the management literature. Fit refers to internal consistency among elements of a research project—research question, prior work, research design, and theoretical contribution. We introduce a contingency framework that relates prior work to the design of a research project, paying particular attention to the question of when to mix qualitative and quantitative data in a single research paper. We discuss implications of the framework for educating new field researchers.

This article is chock full of useful reference charts that can be relied upon to develop a coherent and effective research plan.  For example, this chart [click on to enlarge] summarizes the four key elements of a field research project:

This graphic [click on to enlarge] depicts the iterative nature of good field research:

Two more graphics after the jump:

Testing the Conventional Wisdom of College Football

Empiricism can also used to test hypotheses that matter to actual people.  See, for example, this recent NBER working paper by Trevon Logan (Ohio State Economics):  "Whoa, Nellie! Empirical Tests of College Football's Conventional Wisdom."  Here is the abstract:

College football fans, coaches, and observers have adopted a set of beliefs about how college football poll voters behave. I document three pieces of conventional wisdom in college football regarding the timing of wins and losses, the value of playing strong opponents, and the value of winning by wide margins. Using a unique data set with 25 years of AP poll results, I test college football's conventional wisdom. In particular, I test (1) whether it is better to lose early or late in the season, (2) whether teams benefit from playing stronger opponents, and (3) whether teams are rewarded for winning by large margins. Contrary to conventional wisdom, I find that (1) it is better to lose later in the season than earlier, (2) AP voters do not pay attention to the strength of a defeated opponent, and (3) the benefit of winning by a large margin is negligible. I conclude by noting how these results inform debates about a potential playoff in college football.

Hat tip to Danny Sokol (Missouri Law).

New Book - Tushnet on Judicial Review

Weak Courts, Strong Rights: Judicial Review and Social Welfare Rights in Comparative Constitutional Law

by Mark Tushnet (Princteon 2007)

Availability: Amazon / B&N / Princeton

"[R]ecent innovations in constitutional design in other countries suggest that [social welfare] rights can be judicially enforced-not by increasing the power of the courts but by decreasing it."

Cornell Copyright Duration Chart

Suppose you wanted to assign Herman Pritchett's The Roosevelt Court (1948) in your class. It's out of print. Can you just pirate copy the whole book? In answering this question, a good place to start is the recently updated Cornell Copyright Duration Chart.

There are a couple of simple rules: (1) works published before 1923 are in the public domain, but (2) works published by Disney are not and never will be. For all other works, see the chart.

Empirical Scholarship in the news: mortgages and bankruptcies

Recent stories on American Public Media's Marketplace and in the New York Times highlight the empirical scholarship of Katie Porter of the University of Iowa Law School and Tara Twomey, a Lecturer at Stanford Law School.  The two have been conducting a study, described here, of mortgage claims in Chapter 13 bankruptcy proceedings.  Given the current mortgage crisis, this work has important and immediate real-world implications.

The study, which involved a sample of more than 1000 Chapter 13 bankruptcy filings, found that there were substantial questionable fees charged by mortgage holders.  From the New York Times article:

In an analysis of foreclosures in Chapter 13 bankruptcy, the program intended to help troubled borrowers save their homes, Ms. Porter found that questionable fees had been added to almost half of the loans she examined, and many of the charges were identified only vaguely. Most of the fees were less than $200 each, but collectively they could raise millions of dollars for loan servicers at a time when the other side of the business, mortgage origination, has faltered.

Additionally, the study determined that in about 70 percent of the cases they reviewed, the creditor claimed that the debtor owed more than the debtor thought he or she owed.

This kind of scholarship is, in my view, a particularly exciting example of how empirical legal scholarship can illuminate important matters about how the judicial system actually operates.  And in the current economic climate, this particular study may provide valuable information to regulators, bankruptcy judges, and bankruptcy lawyers alike.