Empirical Legal Studies

Judging National Security Post-9/11: An Empirical Investigation

Cass Sunstein (Harvard/Chicago) has written Judging National Security Post-9/11: An Empirical Investigation.  The Abstract:

Many people believe that when national security is threatened, federal courts should defer to the government. Many other people believe that in times of crisis, citizens are vulnerable to a kind of "panic" that leads to unjustified intrusions on liberty. But to date, there is little information about what federal courts have actually done in this domain, especially in the period after the attacks of September 11, 2001. On the basis of a comprehensive study of relevant courts of appeals decisions in the aftermath of those attacks, this essay offers four findings. First, the invalidation rate is about 15 percent - low, but not so low as to suggest that federal courts have applied a broad rule of deference to government action. Second, the division between Republican and Democratic appointees is comparable to what is found in other areas of the law; contrary to reasonable expectations, there is no significant "compression" of ideological divisions in this domain. Third, and perhaps most strikingly, no panel effects are apparent here. Unlike in the vast majority of other areas, Republican and Democratic appointees do not appear to vote differently if they are sitting with Republican or Democratic appointees. Finally, judicial behavior cannot be shown to have changed over time. The invalidation rate is not higher in recent years than it was in the years immediately following the 9/11 attacks. Explanations are ventured for these various findings, with particular reference to the absence of discernible panel effects.

Exxon Shipping, Footnote 17

Today's New York Times includes,  "From One Footnote, a Debate Over the Tangles of Law, Science and Money,"  by Adam Liptak. 

While failing to give CELS it's full due as THE annual conference on empirical legal studies, it does mention Lee Epstein's keynote speech.  And the article includes pointed quotes from Jeff Rachlinski and Ted Eisenberg on the court's understanding of research on punitive damages. 

Shelton Prize -- Call For Nominations

Steven Willborn, Dean at the Univ. of Nebraska College of Law, recently mentioned that he thought information on the Law School Admission Council's (LSAC) Shelton Prize would interest ELS Blog readers. I immediately agreed and reprint the nomination announcement below. Please note that nominations are due by 1 February 2009 for consideration for the 2009 Shelton Prize. It's nice to see such recognition for and attention to empirical legal research.

Philip D. Shelton Prize for Outstanding Legal Education Research

The Shelton Prize is for outstanding published empirical research related to legal education. The first prize of $5,000 will be awarded in 2009 for research published in 2007 or 2008. The winner(s) will be invited to present their work at the LSAC annual meeting, which will be held in San Antonio, TX, 27-28 May 2009. Winners will be announced in the Chronicle of Higher Education.

Nominations are now being accepted for this prize. Please send a copy of the nominated article and brief information about the author(s) to:

Stephen T. Schreiber
Executive Vice President and Chief Operating Officer
LSAC
662 Penn Street
Newtown, PA  18940
[email protected]

Nominations must be received by no later than 1 February 2009 to be considered for the 2009 Shelton Prize.

Data Update: New ICPSR Data on Hate Crimes

The ICPSR has just released Uniform Crime Report data on hate crimes in record-type format for the years 1996-1999.  Together with two earlier releases, this means that there are now available incident-level data on hate crimes from 1996-2006, inclusive.

"Never begin a project with an open mind"

What do you do with a null results dataset?  A colleague wanted to study trends in tortious interference cases in the U.S. over the last 20 years.  He had some firm priors but was going to conduct the study with an open mind (hence the title of this post).  We designed a data collection protocol, pulled a sample of 100 cases (an adequate number given his priors) and set an RA to work.  Nothing.  We redesigned the protocol, pulled another sample, set another RA to work, and again nothing.  After 8 months there are no meaningful temporal or cross-sectional variations in the data.  These are all civil jury trials so there may be a story about the continuity of juries.  But he's not that interested in collecting more data, or even writing up what he's learned so far.  No one wants to read about what he didn't find. 

There is a publication bias against negative results.  Michael Heise blogged about it here a couple of years ago.  Attempts have been made to unbias the bias, but success is limited.  Most of us are probably not in the mood to finish a 50+ page paper with null results, so the idea of nurturing an ELS journal or SSRN site for that purpose is not high on my list.  The data, on the other hand, are finished: collected, cleaned,  documented.  There is nothing wrong with them, yet they are orphaned on a hard drive.  Is there a data orphanage?

Judicial Decisionmaking "By The Numbers"

Those interested in the growing empirical literature on judicial decisionmaking will want to read a thoughtful paper by Brian Tamanaha (St. John's). The Distorting Slant of Quantitative Studies of Judging develops two main points:

"The first point is that the field was born in a collection of false beliefs and misunderstandings about the formalists and the realists which has distorted how political scientists have modeled judging and how they have designed and interpreted their studies. Rather than conduct an open inquiry into the nature of judging, political scientists set out to debunk formalism by proving that judging is infused with politics, a mission that warped the development of the field.

The second point is that the results of their studies below the Supreme Court strongly confirm what judges have been saying for many decades - that their judicial decisions are substantially determined by the law. Political scientists have tended to repress this finding, however, by focusing on the wrong point: repeating time and again that their studies show that politics matters without also emphasizing that it matters very little. A balanced realism about judging accepts that - owing to the uncertainty of law and the inherent limitations of human decision makers - it is inevitable that there will be a certain (minimal) degree of political influence in judicial decision making, but this does not detract from the broader claim that judges can and usually do rule in accordance with the law."

Gretl

The ideal statistical software for teaching would be powerful, easy to learn and free.  Stata, SPSS and SAS are powerful and relatively easy to learn, but not exactly free.  R is free and very powerful, but has a long-toed learning curve (see previous discussion on software package preferences  here).  At one point I toyed with PSPP (an open-source knock-off of SPSS) but it is a very poor substitute and not worth anyone's time.  Now I've found Gretl (The GNU Regression, Econometric and Time-Series Library).  Gretl is a stand-alone open-source cross-platform package.  It can directly import files from Stata 9, Excel, csv and several other formats, although the imports do not always proceed smoothly.   It uses menus or scripting, and it can invoke R  for more sophisticated analysis and graphing (if R is installed).  The built-in routines include the expected (e.g., ols and logit) as well as time-series and some arcana (3sls).  The post-estimation analyses are first rate.  Gretl won't supplant Stata on my personal desktop any time soon, but it's worth considering for the classroom if you are teaching statistics to law students with Excel or Datadesk. 

Research Conference on Access to Civil Justice: Empirical Perspectives

Yesterday was the best of days, and it was the worst of days.   I was a participant and presenter at the "Research Conference on Access to Civil Justice: Empirical Perspectives," which is an ambitious event organized by the indomitable Sam Estreicher, NYU law prof and co-director for the Opperman Institute of Judicial Administration. 

The good part of the day was being surrounded by intelligence people of good will who want to empirically study, and ultimately improve, our various systems of civil justice, particularly for poor and working class individuals.   The bleak (or sobering) portion of the day was coming to grips with (a) how few Americans can obtain the services of a lawyer to remedy serious wrongs, (b) how poorly the system often operates, even if you have a competent lawyer, and (c) how this process can foment profound misunderstandings, disappointment, and disengagement among the very people that the law is putatively supposed to protect.   

Here is a quick thumbnail sketch of  the key takeaways from each presentation (note these are my intepretations, not necessarily the views of the researchers):

• Gillian Hadfield (USC Law), "Assessment of Legal Needs":  A large proportion of poor and middle-class Americans households (25 to 40%) with a perceived legal problem take no steps and contact no one; this is much higher than places like Scotland (3%), England & Wales (5%), and Slovakia (18%).  The proportion of contact with lawyers is roughly the same in all these countries (20% to 30%), but third party providers and institutions seem to play a much larger role in achieving a resolution.  One plausible reason for these patterns is that typical Americans are more likely to accept that the system cannot or will not provide them with relief.

• Laura Beth Neilsen (Northwestern Sociology/ABF), "Pro Se Claimants in Federal Court: An Empirical Analysis of Legal Representation in Employment Civil Rights Cases":  Drawing upon a fabulous, large, multi-city dataset of federal employment discrimination cases, we learned that (a) employment discrimination claims are the most common type of civil claim in federal courts; (b) they are disproportionately filed pro se; (c) net of other factors, pro se are more likely to be dismissed; (d) even when a plaintiff wins (because of conduct that can shock the conscience) the process itself can exact a huge personal and emotional toll, which can exacerbate perceptions of discrimination and alienation among protected groups.

• Rafael Pardo (Seattle Law), "An Examination of Access to Chapter 7 Relief by Pro Se Debtors":  After assembling a large sample of Chapter 7 bankruptcy cases covering a time period both before and after the 2005 federal bankruptcy reform (which heightened the law's complexity), we learned about the large proportion of pro se cases (~20%).  In turn, Rafael used multi-regression analysis to show (a) that filing pro se is associated with a much higher level of case dismissals, (b) that the change in the law was not associated with heightened rates of dismissals generally, but (c) the interaction between pro-se and filing under the new BAPCPA statute did lead to higher dismissal rates.  This raises the question of whether folks can afford to pay for the law's heightened complexity.

• Joy Radice (NYU Law), "Expanding Access to Civil Legal Services: Four Profiles of Legal Aid Organizations Serving Low-Income Communities":  Roy used the case study method to develop commonalities and differences among four examples of legal aid to low-income communities.  The presentation revealed the vagaries of depending upon the Corporation for Legal Services funding (including curtailment of professional discretion) and how threadbare these operations really are, often having affects on case selection.  One exception to the general findings was a legal aid center staffed by Harvard Law students, which had the benefit of careful planning through an elite law school.  But clinics staffed by law schools, no matter how outstanding, are not a template for the larger problem of lack of access.  The emphasis in some organizations on litigation over ADR (especially meditation) was a little surprising, though I need to be careful about generalizing from case studies. 

• Andrew Morriss (Illinois Law), "Clinical Legal Education: Evolution of Law School Clinical Programs":  Andy discussed how clinics evolved out of (a) a perception among the bar that law graduates lacked adequate skills (sound familiar) and (b) huge infusions of Ford Foundation money in the 60s and 70s, which arguably was a tipping point in wider adoption of clinical programs.   Andy created a typology of clinics and presented how they broke out by relative rank, with "impact" clinics most prevalent at high prestige schools.  He also presented evidence that "impact" clinics as opposed to "access" clinics were part of larger project for clinicians to gain status relative to their tenure-track peers--a project that is pushing toward an elimination of the doctrinal/clinical distinction.   Be that as it may, Andy suggests that access for the indigent--one of the original purposes of clinics--is itself becoming marginalized at many institutions.

• Jon Nash (Emory Law), "What Do Pro Bono Programs Do?":  This presentation reviewed the plausible theoretical drivers for large law firm commitments to pro bono--e.g., training for associates, a recruitment tool, burnishing of brand, all of which could potentially redown to the bottomline.   Drawing upon data from Law Firm Working Group, which has firm attributes and average pro bono hours per lawyer going back 15 years, there is weak positive support for the business case.  When the data are disaggregated by firm HQ location, however, the a simple business story seems far less plausible.  Going back to the access theme, the paper prompted the larger question of what types of pro bono these firms were doing.  Publishing pro bono hours in The American Lawyer may itself be a driver of more pro bono work.

• Hon. Robert Katzmann (Second Circuit Court of Appeals) then delivered a final keynote address entitled "The Unmet Legal Needs of Immigrants: The Role of the Legal Profession."  Judge Katzmann reported on the veritable flood of immigration appeals since the BIA moved toward more summary procedures to reduce its own glut of cases.  More aggressive immigration enforcement in the post-9/11 world also may be a factor.  Remarkably only 35% of immigrants in his court's docket have the benefit of legal counsel.  In addition to be terrifying experience of being a poor indigent person in bureacratic government maze in a new country, the lack of representation can have disasterous personal consequences, including deportation and separation of families.   Solutions to this serious lack of access problem--none of them easy or quick or entirely satisfying--were then discussed.

Today we have several more presentations from highly skilled empiricists.  I am looking forward to the final plenary session, where we can discuss some action items.

An ELS toolbox

My thanks to Bill Henderson for his introduction, especially the last paragraph.

I am not a lawyer but a political scientist employed by a law school. My perspective on ELS is somewhere between a carpetbagger and a transplant.  I share with my collaborators an interest in the law, but I am much more fascinated by the various models of human and institutional behavior implied by our research questions.  Each one allows me to reach into my toolbox and pull out methods rarely used in political science, such as path analysis, structural equation modeling, network analysis or Weibull regression, in addition to probit, logit and ols.  (I am fortunate to have access to the UCLA Statistical Consulting Group when I feel shaky ground.)

The variety of techniques that I and others deploy in ELS has me wondering about the processes by which other disciplines narrowed their toolsets.  When I was in graduate school sociologists used ANOVA, economists used maximum likelihood and political scientists used logit and ols.  There were some crossovers, but those were the exceptions.  The methods in those fields have evolved as economists invaded expanded their portfolio into the other disciplines, and the fields became more statistically sophisticated.  Hiving persists nevertheless, and I suspect that this is because the metrics are mature and research design is cumulative.  Introducing a new technique to a field is not merely a question of teaching the methodology but also of gaining a foothold in the literature.  The slow-motion adoption of social network analysis into political science is a good example. It's relatively easy to measure a network of individuals engaged in governing, but the language necessary to describe it as a political party is still evolving.

Which leads me to wonder about the utility of having a set of metrics and tools unique to  ELS.  An ELS toolbox would be the dialect of empirical legal studies, aiding the transmission of knowledge within our group while setting boundaries around what constitutes our field.  Is this likely to happen?  Probably not.  We are not isolated enough.  We don't have graduate students in the traditional sense, and our mandate is to produce new professionals not clones of ourselves.  So we are in an odd position, building a field that is defined by the use of social science research methods, but without a set of methods to call our own and no prospect of creating one.  This might be an advantage, as it gives us the liberty to borrow from everywhere, but it leaves open the question: what defines ELS? Is it what we study, how we study it or who does the studying? 

Guest Blogger Joe Doherty

The ELS Blog is pleased to welcome guest blogger Joseph Doherty, who currently serves as Director the UCLA Empirical Research Group (ERG).  Joe has a Ph.D. in political science from UCLA and is an expert in quantitative research design and methods. Through ERG he works with the UCLA School of Law faculty to conceptualize, design and execute empirical research across a broad range of legal fields.  Joe has advised or collaborated with scholars in studies that include bankruptcy, legal aid, administrative law, pollution prevention, tax policy, gay rights, the living wage, campaign finance disclosure, criminal law and immigration policy.  He has co-authored research on bankruptcy with Lynn LoPucki, on the living wage with Rick Sander, and on administrative law with Jody Freeman.  In addition, Joe is leading the new UCLA-RAND Strategic Alliance for Law and Public Policy, a collaborative project between the two institutions that includes research on corporate law and civil justice. 

Joe also specializes in voting behavior and campaign finance. From 2002 to 2006, he served as Project Director of the Campaign Disclosure Project (CDP).  In this capacity, Joe created an online database of state-level campaign disclosure laws and developed an innovative method for measuring the usability of campaign finance disclosure websites across states.  The CDP is a collaborative effort of the UCLA School of Law, the Center for Governmental Studies and the California Voter Foundation, and is funded by The Pew Charitable Trusts.  It has been credited by legislators and activists across the country for providing the impetus to modernize state laws and regulations, and to make more and better campaign finance data available on the internet. 

Finally, among empiricists in the legal academy, Joe is known as a good person who is always willing to offer this time and expertise to fellow researcher.  We look forward to Joe's comments.

The Empirics of Estate and Inheritance Taxation

Those inclined to look at something other than a steadily plunging Dow might consider Dead or Alive: An Investigation of the Incidence of Estate and Inheritance Taxes. In the paper, the authors, Lily Batchelder (NYU) & Surachai Khitatrakun (ERS Group), consider three questions: "(1) the relative burden of wealth transfer taxes on heirs versus donors in a partial equilibrium context, (2) the distributional effects of the estate tax if its burdens are assigned to heirs, and (3) whether the incidence of a wealth transfer tax fundamentally differs depending on whether it is based on the amount transferred (an estate tax) or the amount received (an inheritance tax)."

The paper argues that, "contrary to existing practice, that it is more accurate to allocate wealth transfer tax burdens to heirs as a rough first approximation. It then presents the first estimates of the distribution of federal wealth transfer tax burdens based on this assumption. It finds that the 2009 federal estate tax is highly progressive by various measures of economic income if its burdens are assigned to heirs. It also finds that the estate tax contributes importantly to the progressivity of the tax system overall by partially offsetting the exclusion of inheritances from the income tax base among high-income heirs."

The Globalization of the Legal Profession

For anyone interested in the  economic and political forces that are reshaping the legal profession, here is an event that you need to attend.  On Friday, November 21, the Harvard Law School Program on the Legal Profession and the American Society of International Law are sponsoring a conference entitled "The Globalization of the Legal Profession."  Conference registration is free, but space is limited.  Details are online here.

Back in April, the Center for the Study of the Legal Profession at GULC put on a conference entitled the "Future of the Global Law Firm."   Similar to upcoming program at Harvard, speakers included academics, practitioners, clients, and allied professionals.  The shared sense of rapid and major structural change was palpable.    I consider the upcoming program an important extension of that conversation. 

Further, as I reflect on the proliferation of so-called "global law schools" in China and India, which are being set up to serve major US and UK legal employers (and a topic at the HLS event), I am convinced that globalization will eventually reshape the  American legal education system.   Now is the time to plan; in ten years, the majority of law schools could be in the uneviable position of reacting to a major structure change.

More From BJS: State Civil Litigation Trends, 1992--2005

Supplementing one of the leading sources of data for state civil litigation activity in the United States, the Bureau of Justice Statistics (BJS) recently released a report, Civil Bench and Jury Trials in State Courts, 2005, which provides an important snapshot and illustrates litigation trends since 1992. The report discusses general civil cases (tort, contract, and real property) concluded by a bench or jury trial in a national sample of jurisdictions in 2005. Topics include the types of civil cases that proceed to trial, the differences between civil cases adjudicated by judges or juries, and the types of plaintiffs and defendants represented in civil trials. Also, the report covers plaintiffs.  Key findings include:

• In 2005 plaintiffs won in more than half (56%) of all general civil trials concluded in state courts. The plaintiff was significantly more likely to win in a bench trial compared to a jury trial. Among all plaintiff winners the median final award was $28,000. Approximately 4% of all plaintiff winners won $1,000,000 or more. Contract cases in general had higher median awards ($35,000) than tort cases ($24,000).

• The total number of civil trials declined by over 50% from 1992 to 2005 in the nation’s 75 most populous counties. Tort cases decreased the least (40%) while real property (77%) and contract (63%) cases registered the largest declines.

• In the nation's 75 most populous counties, some tort case categories have seen marked increases in their median jury awards. This was particularly the case for product liability trials, where the median awards were about 5 times higher in 2005 than in 1992 and for medical malpractice trials, where the median jury awards more than doubled from $280,000 in 1992 to $682,000 in 2005.