Empirical Legal Studies

Empirical study linking economics training to higher income as lawyers

Josh Wright over at Truth on the Market links to a study that I have heard about (and been skeptical of), but had not yet seen in print:  “Do Economists Make Better Lawyers? Undergraduate Degree Field and Lawyer Earnings.”  The comments float some rival hypotheses.  In the hope of seeing a novel research design, I am adding this paper to my summer reading pile.

One reason why I am interested in this topic is that labor markets for new lawyers appear to follow a pretty simple pattern:  higher paying jobs go to students with higher grades from higher ranked schools.   (For evidence that geography also plays a key role, see Henderson & Morriss (2006)).  I know of only limited anecdotal evidence that employers are willing to forgo school rank or grades for specialized substantive training (e.g., I have heard persuasive accounts from two law profs that employers will dig a little deeper into the class during OCI for students who completed a business law certificate program; another example is the $15K bonuses being paid at some firms for JD/MBA grads). 

Imagine how the law school universe would be turned on its head if it could be shown empirically that individual law schools, through a well-designed curriculum, added sufficient enhancements in human capital that prestigious employers would trade down a few notches in USNWR (which correlates at .90 with median LSAT scores) to hire those graduates.  If anyone has additional anecdotal evidence on this issue, please post a comment.

Wiener on Lott v. Levitt

Today's issue of the L.A. Times includes an op-ed piece on Lott v. Levitt by Professor Jon Wiener of U.C. Irvine's history department. (Credit to Deltoid and Kevin Drum for earlier postings.)

Courts Serving Party Coalitions

It is not uncommon for legal scholars to conceptualize courts as acting in opposition to politics -- as restraining or checking the exercise of power.  On this view, it is reasonable to assume that, in most cases, the exercise of judicial power comes at the expense of other power holders.   

Over the last few years a number of scholars, working under the rubric of a "regime politics" approach to the study of courts, have flipped this assumption in order to explore how much of the behavior of judges -- and (for that matter) how much of the development of law -- can be explained if we assume that courts often serve what Robert Dahl once called the "dominant governing coalition" of a polity (see "Decision-Making in a Democracy: The Supreme Court as a National Policy-Maker," Journal of Public Law 6 (1957): 279–295).  It is a starting point that is also consistent with Martin Shapiro's argument (developed in his book Courts:  A Comparative and Political Analysis, among other places) that we should think of courts as inevitably constructed to advance the interests of the regime with which they are affiliated.   (All of this is premised on the assumption that the staffing and jurisdiction of courts is typically controlled by other power holders who are not exclusively motivated by concerns about competence and integrity.)

What roles might courts play in promoting the political agenda of dominant governing coalitions? 

Law and Strategy

Arriving in the mail this weekend -- the brand new co-edited volume by Ronald Kahn and Ken I. Kersch, The Supreme Court and American Political Development (Kansas).  Congrats to Ron and Ken.  (Full Disclosure:  I have a contribution in the volume; despite that, they still deserve the kudos.)  It's highly recommended to anyone interested in exploring so-called "APD" approaches to the study of the Court. 

One brief shout-out that might be especially relevant to readers of this blog:  those of you with an interest in debates about "attitudinal v. strategic v. legal" approaches to Supreme Court decision making will want to read Mark Graber's chapter, entitled "Legal, Strategic, or Legal Strategy:  Deciding to Decide during the Civil War and Reconstruction."  Among other things the chapter calls into question the view that Ex parte McCardle is best understood as a strategic retreat.  More generally (to quote his last sentence), Graber tries to show that "the most fruitful investigations will explore the ways in which legal, strategic, and attitudinal factors interact when justices make decisions...."

Empirical

Michael Klarman is a legal historian who likes to explore the kinds of questions that political scientists ask about the Supreme Court.  In his extraordinary book From Jim Crow to Civil Rights he addresses a topic that has been dear to the hearts of many judicial behaviorists:   are Supreme Court decisions motivated by legal considerations or by non-legal considerations, such as the justices' personal attitudes or other political calculations?  When behaviorists take on this question they begin by developing measures of law and judicial attitudes (e.g., see here and here).  Klarman opted for an in-depth case study (see also his draft essay here).

Specifically, he identified the justices who participated in Brown v. Board of Education and collected evidence regarding their personal, political, and legal views on segregation.  Not surprisingly, Klarman found that many of the justices who personally considered segregation to be immoral also adopted legal positions that treated segregation as unconstitutional.  In some cases the data were ambiguous, and the best Klarman could do was offer informed speculation -- for example, as to whether Cold Warriors Burton and Minton (who had not previously exhibited support for the active judicial protection of civil rights and liberties) sincerely believed that segregation violated equal protection or were instead swayed primarily by the Cold War imperatives of the case.  Klarman also made a very persuasive case that, for Justices Frankfurter and Jackson, Brown represented a clear clash between legal considerations (which, in their judgment, argued in favor of upholding Plessy) and personal/political considerations (which they believed argued in favor of overruling Plessy).  The overall result was a richly textured discussion of how personal, political, and legal factors influenced the outcome in Brown.

Guest Blogger: Howard Gillman

Our guest blogger this week is Howard Gillman. Howard is a Professor of Political Science, History, and Law, and Associate Vice Provost for Research Advancement at the University of Southern California.  He specializes in constitutionalism, the U.S. Supreme Court, and judicial politics. 

His most recent book is The Votes that Counted: How the Court Decided the 2000 Presidential Election (University of Chicago Press, 2001). His first book, The Constitution Besieged: The Rise and Demise of Lochner Era Police Powers Jurisprudence (Duke University Press, 1993), received the C. Herman Pritchett Award for "best book in public law" from the Law and Courts section of the American Political Science Association. He is also co-editor and contributor to two other books on the Supreme Court: Supreme Court Decision-Making: New Institutionalist Approaches (University of Chicago Press, 1999) and The Supreme Court in American Politics: New Institutionalist Interpretations (University Press of Kansas, 1999).

New Book - Judging on a Collegial Court

I don’t think we have highlighted the recent publication of Judging on a Collegial Court: Influences on Federal Appellate Decision Making by Virginia Hettinger, Stefanie Lindquist, and Wendy Martinek (University of Virginia Press). I am happy to do so now.

The Polls - NSA Surveillance

Mystery Pollster Mark Blumenthal comments on the various polls here on the CBS News PublicEye blog (rather than his own blog), and he provides links to the full results from each poll. It's a good discussion, though I question his final comparison of two poll items in the penultimate paragraph.

As a side note, view the National Security Agency Kid's Page here. Apparently, this has been around for at least a few months. Somehow, I missed it. The NSA says it's fun for adults too.

White, Male SCOTUS Clerks

The Legal Times has a interesting story on the lack of ethnic and gender diversity among U.S. Supreme Court Law Clerks.  The story draws heavily on the research of former ELS Guest Blogger, Todd Peppers, who participated in the "Law Clerks as Research Subjects" Forum back in March.  Here is a worthwhile excerpt that deals with changes in hiring practices over time:

This is ...  the first term in which clerks affected by the new federal clerkship hiring plan have reached the Court. Under the plan, adopted in 2002, appellate judges agreed they would no longer hire clerks after their first year of law school but would wait until the start of their third year. Some judges speculated that by giving law students another year to blossom, more minority candidates could emerge. If so, it did not result in more minority clerks at the Court.  [Only three of thirty-seven this term are non-white.]

Todd Peppers, author of the new book "Courtiers of the Marble Palace," which takes a historical look at high court clerks, says none of the possible explanations for low numbers of minorities is satisfying. When, a few years ago, members of Congress asked why justices could not cast their nets more widely to find more minority candidates, Peppers recalls, the justices often fell back on the "questionable" excuse that they could not afford to take chances that even one of their clerks might not be a top performer.

2006 Conference on Empirical Legal Studies

Click here for complete information on the inaugural Conference on Empirical Legal Studies ("CELS").  This website will assist those interested in attending the conference, submitting a paper for consideration (paper submission deadline=June 30, 2006), and conference participants and attendees seeking logistical and registration information. 

The 2006 conference will be held at the University of Texas School of Law in Austin, Texas, on Friday 27–Saturday 28 October 2006.  For those who really like to plan ahead, the 2007 and 2008 conferences will be held at New York University School of Law and Cornell Law School, respectively.

Wright and Engen on Prosecutor Power

Ronald Wright (Wake Forest) and Rodney Engen (NC State) have written an empirical analysis of prosecutorial discretion entitled The Effects of Depth and Distance in a Criminal Code on  Charging, Sentencing, and Prosecutor Power.  The Abstract is below the fold:

A Recap of the Empirical Legal Scholarship Workshop

As I stated earlier, I just attended the 3-day Empirical Legal Scholarship Workshop held at Northwestern in Chicago. It was great to see such diversity among the attendees in terms of substantive areas of interest, level of experience doing empirical work and understanding of statistics, and variety of law schools. The workshop is clearly geared for beginners, though it provides a good statistics refresher course for those of us who already engage in empirical work. (Though it looks like there will be an “Advanced” workshop held at Wash U next academic year that presumably will start to dive into the various complexities of multiple regression.  Andrew asked for suggestions on topics to cover.  My list included panel effects, rare events, graphics, time series, diagnostics, and predicted probabilities, among other items.)

 A number of basic and helpful tips were delivered at the workshop. For example: (1) Do not be an advocate for your hypotheses.  Always look for alternative accounts. (2) Write down every single way that you could measure or operationalize a concept. (3) Andrew Martin justified the use of STATA as the preferred stats software. Why STATA? More control over graphics; fewer limitations of stats models than SPSS; really cool modules for STATA; flexible between menu and commands; less $ than SPSS. But he made clear that ease is not one of the reasons for picking STATA.  (The ELS Blog reached a similar conclusion.) (4) If you take a stats course in the future be sure to learn the following: Diagnostics for outliers and leverage; Diagnostics for non-linearity; Diagnostics for incorrect distributional assumptions; Multi-collinearity; Models for different types of dependent variables.

Day one started with a discussion of the Sunstein et. al piece about ideological voting patterns on the courts of appeals, serving as the introduction for Lee Epstein’s conceptual overview of empirical work, research design, data collection and coding. Days two and three, mostly taught by Andrew Martin, was, as he called it, an overview of the first 2/3 of an intro stats class (inference, descriptive stats, cross-tabulations, linear regression, etc.), only introducing multiple regression. There were some classic objections to the Sunstein piece: the piece did not look at judicial reasoning in opinions, no operationalization of law, and frustration with the explanations of findings, though not as much discussion about inter-coder reliability as I would have expected.   In addition, a helpful paper “Coding Variables” by Epstein and Martin (Encyclopedia of Social Measurement, Vol. 1, 2005) was also distributed, in addition to several other helpful handouts on STATA commands, statistics vocabulary, and helpful references.  Overall, a worthwhile program.

Reports from the Stats for Law Profs 'Boot Camp'

In addition to Jason's observations (and recap), over at the Conglomerate blog, Christine Hurt (Marquette/Illinois) has posted some thoughts (here and here) on the recently completed  Empirical Scholarship Workshop.  By all accounts political scientists Lee Epstein (Wash U./Northwestern) and Andrew Martin (Wash U.) -- both future ELS guest bloggers -- run a great program consciously designed to help law profs and legal scholars transition into empirical scholarship.

The Value of Stats for Law Students and the Legal Profession

Over at Prawfsblawg, Ron Wright has an absolutely terrific post on constructive steps toward a legal profession that can take full advantage of empirical research and statistics.  The comments are also worth reading including a comment on the value of a law school stats course.

New Empirical Legal Scholars

I just attended the Empirical Scholarship Workshop (which I'll blog more about later).  It was exciting to see so many participants who for the first time are engaging empirical legal studies.  At the end of the program a number of participants wondered what they could do to continue to educate themselves about empirical methodologies (e.g., texts, classes, etc.).  We've posted on this general issue at least four times in the past, so I thought it would be beneficial to link to these posts all at once for any potentially new readers that I met at the workshop.  See here (helpful texts), here (stats software), here (doing ELS for the first time), and here (doing ELS without formal training, e.g. a PhD). I encourage readers to also view the comments as they are quite informative.

Most Heavily Cited Court Cases

On the Tax Prof Blog, Paul Caron reports on the recent work of his colleague, Adam Steinman, who documented the most heavily cited court cases in his discussion of Celotex (a Civ Pro I standby).   Paul's blog entry includes a table of the Top 10.  Seeing no tax cases, Paul generated his own Tax Top 10, which he also posts.

All this reminds me of the fascinating network analysis of court cases by Tom Smith of the University of San Diego Law School.  (See, e.g, The Web of Law.)   There is a lot of work to be done in this area.  The digitization of the data + adequate computer power are relatively new phenomena.

Federal Data Quality Revisited

I earlier posted about agency guidelines in reponse to the Federal Data Quality Act.  Lauren Willis (Loyola-LA) wrote a letter in an attempt to use the OMB Guidelines to critique an Office of Comptroller of the Currency proposed regulation change that relied heavily on an OCC Working Paper that used a data sample to come to the conclusion that subprime home loans are priced efficiently.  See here.  Though it seems there was no response to the Willis critique.  Apparently, in publishing the final regs, the OCC included statements about how it was complying with OMB requirements, but failed to mention the OMB Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies.

Administrative Office Data

It is my great pleasure to be a guest blogger on the ELS website.  Kudos and thanks to Jason and his colleagues for establishing this terrific site, chock full as it is with interesting information for all of us interested in ELS. 

In this first entry, I thought I'd address a question that may be of interest to many readers about the Administrative Office Database, now archived in computer readable format by the Federal Judicial Center and available through the ICPSR.  Ted Eisenberg has noted that some have called the AO database "'by far the most prominent' database used by legal researchers for statistical analysis of case outcomes"  (78 Notre Dame L.R. at 1458).  One frustrating thing about the AO database, as we all know, is that it has the judge ID's expurged prior to its distribution to the public.  My understanding of this situation, learned during my stint at the FJC, is that the decision to release the data without the judge identification information was initially rendered by Chief Justice Rehnquist in consultation with the AO.  The concern was that statistical analysis of individual judges' decisions would threaten judicial independence and possibly even the safety of judges as well.  Nevertheless, this decision led to the paradoxical situation in which the NSF ultimately spent hundreds of thousands of dollars to fund the Songer Database (the U.S. Courts of Appeals Database)--although no similarly publicly funded database for the District Courts has yet to be developed.  Although the Songer Database includes information not available in the AO data, it would have been far simpler for Don to have had access to the judge codes as an important starting point in the development of his database.

At this point, it seems the time might be right for the scholarly community to petition the federal judiciary to release the judge codes based on several prevailing factors.  First, we have a new Chief Justice who may be more willing to consider such a request.  Second, sophisticated computer programs now exist to gather vast amounts of information directly from Westlaw, including the identity of judges rendering the decisions.  These computer programs have been used, for example, to produce complex network analyses of precedent based on thousands of cases in the Westlaw database (Jim Spriggs and his colleagues have initiated such analyses, for example).  Third, the Songer data exists in the public domain with considerable information about the decisional patterns of individual judges readily available.  Fourth, the ludicrous situation in which the federal government gives with one hand while taking away with the other--at a high cost to taxpayers--might be a compelling argument in support of such a request.  Finally, from a scholarly viewpoint, analyses of case level decision making without controls for ideology make little sense given the state of our knowledge about attitudinal factors in judicial decision making.  Moreover, the AO data includes information about unpublished opinions--presently not available in the Songer database.

One problem, of course, is that the federal judiciary is exempt from FOIA.  Thus, the only way to obtain the information is for us to petition the AO to release it.  The Executive Committee of the APSA Law and Courts Section might be a place to start in the development of such a request, but I welcome suggestions from readers, who might have other insights into this issue. 

Practical Empirical Research Project Management Tips

A recent thread on the SPSS listserv begins with: "One continuing problem everywhere I've worked is trying to go back to an analysis someone else did 2 years ago and reconstructing what they did, especially regarding data cleaning and other changes to the data files."  This is a real problem with immediate, important, and practical implications (e.g., replication).  The SPSS thread has already generated a few tips (some more useful than others) that I pass along (in summarized form).

General:

1.  "File system."  For each project set up 4 subfolders (setup, data, raw data, and analyses).
        a.  Setup -- syntax file, plus other project documentation (e.g., codebook).
        b.  Data -- master data file along with subfiles.
        c.  Raw Data -- original raw data.
        d.  Analyses -- analyses syntax files (but don't save output files if disk space is an issue).

Syntax:

1.  Save and date all syntax files.  Annotate syntax files with comments, space out the code neatly, and make sure the log file is turned on.

2.  Create a syntax file for your data setup (i.e. reading, converting, tidying, labeling, setting missing vals, formatting).

3.  Create separate syntax files for separate analyses.

Data:

1.  Create a new data file to reflect any major change to data.

2.  Give data and related output files distinctive and sequential names.

3.  Link names for syntax and outfiles with name of source data file.

I welcome other suggestions.

Lott v. Levitt: A Quick Update

HarperCollins and Steven Levitt were served on May 2, 2006 and May 5, 2006, respectively. HarperCollins’ response was originally due today; Levitt’s response was originally due this Thursday. However, the court granted the defendants’ motion for an extension to Monday, June 5th, and neither defendant filed a response today.

Can Women and Minority Lawyers Duplicate the 'Jewish Law Firm' Success Story?

The New York Law Journal recently published a story with the provocative title, "Can the 'Jewish Law Firm' Success Story be Duplicated? "

The article highlights the empirical research of Eli Wald (Denver Law), who is doing some interesting work on the rise of Jewish lawyers within the  New York corporate bar.  Wald's research includes interviews with numerous Jewish lawyers who joined large law firms between 1946 and 1962, essentially breaking a de facto ethnic barrier.  Eventually the taboo of hiring Jewish lawyers exploded in the 1970's when predominantly Jewish law firms, such as Skadden Arps, Weil Gotshal, Paul Weiss, Fried Frank, Kaye Scholer, and Wachtell Lipton, became the counsel of choice for corporations seeking innovative solutions to old problems.  (For an eye-opening contemporaneous account of the pre-1970s discrimination, see Notes and Comments, The Jewish Law Student and New York City Jobs -- Discriminatory Effects in Law Firm Hiring Practices [Hein Online], 73 Yale L. J. 625 (1964)).

According to Wald, the bottom-line is that the Jewish success story reflects a unique historical occurrence that is unlikely to be duplicated by female or minority lawyers.   Bruce MacEwen, who writes the Adam Smith, Esq. blog on law firm economics, has a very informative and insightful analysis of Wald's findings.  Both the NYLJ story and MacEwen's commentary are recommended reading.

Who Wins Patent Infringement Cases?

A new article by Paul M. Janicke & LiLan Ren, Who Wins Patent Infringement Cases?, 34 AIPLA Q.J. 1 (2006), offers some data on patent infringement cases in the Federal Circuit. While there is no abstract, the following text from the introduction provides the gist of the article:

Numerous studies in recent years have focused attention on particular aspects of patent infringement litigation in the United States.... No one, however, seems to have analyzed the results in terms of who ultimately “wins” patent litigations, meaning a judgment in favor of a particular party at the Federal Circuit Court of Appeals, for the full population of contested judgments. We undertook to fill that void by analyzing the dispositive decisions of that court over the three-year period from 2002 to 2004. By “dispositive” we mean a case in which, as it leaves the Federal Circuit, at least one claim of one patent is finally adjudicated to have been infringed and not invalid or unenforceable (i.e., a win for the patent owner), or in which it has been finally determined that no claim has these characteristics (a win for the accused infringer). We included all dispositive decisions of the Federal Circuit, whether by precedential opinion, non-precedential opinion, or per curiam affirmance without opinion under the court’s Rule 36. There were 262 such cases.

Below is a sample of their results, which reports how often each judge voted for the patentee. (Note: In reproducing this table, I moved Judge Lourie up two rows to maintain the descending order in the fourth column.) The Federal Circuit randomly assigns judges to panels.

The Difficulty of Modeling Leadership and Sincerity (or Why Democrats Might Fail Again in 2006)

I know this is an academic blog (as opposed to a “mixed” blog). Thus, we generally don’t discuss politics. But several weeks ago, I was party to some lively dinner conversation that raised an interesting issue of the limitations of empirical research for political consulting. So, despite the political context, this is a serious—very “serious”—blog post, assuming there is such a thing. Further, it is probably right in the wheelhouse of some political scientists out there who specialize in voting and election strategy.

Here is my question: Is it possible that the constructs of leadership or sincerity, which cannot be easily measured, are part of the error term in the voting models used by political consultants? In other words, does the conventional political wisdom rest upon a series of mis-specified regressions? The context of my dinner conversation might throw this issue into sharper relief. 

My companions included an economist, a historian, and two political scientists.

Should Law Students Take a Stats Course?

See one answer here.

Guest Blogger: Stefanie Lindquist

Our guest blogger this week is Stefanie Lindquist (Vanderbilt).  (Her CV is available here.)  Her publications include:

Judging on a Collegial Court: Influences on Appellate Court Decision Making. With Virginia Hettinger, Wendy Martinek. University of Virginia   Press, forthcoming 2006.  Click here for publisher's abstract.

“The Influence of Jurisprudential Considerations on Supreme Court Decision Making: A Study of Conflict Cases,” Law and Society Review, 40(1):135-162 (with David Klein).  

 "Comparing Attitudinal and Strategic Accounts of Dissenting Behavior on the United States Courts of Appeals." 2004. American Journal of Political Science 48(1):123-137 (with Virginia Hettinger and Wendy Martinek).

"Separate Opinion Writing on the United   States Courts of Appeals," 2003. American Politics Research, 31(3):215-250 (with Virginia Hettinger and Wendy Martinek).

Article ranking U.S. law reviews

Over at Conglomerate, Gordon Smith posted an excerpt from a new ranking system of American law reviews by Ronen Perry of the University of Haifa (Israel).   The excerpt includes a summary table of the Top 25 journals.  Perry's system looks at overall citations of articles published and citations per page and thereafter normalizes to a 100 point scale for ranking purposes.  According to Perry, "The linear correlation coefficient between the school's ranking and its flagship law review's ranking is 0.8334." 

No surprise there.  Many law professors use U.S. News rankings as a guide during the submission and expedite process.  One reason why the correlation is not higher is that a handful of articles at non-elite journals probably garnered a large number of citations (i.e., they were especially good, novel, or bad in a truly noteworthy way).  Presumably, this effect is random.   But there is no sound theoretical reason to believe that any alternative ranking of law reviews (whose staffs turn over every year) is going to be valid prospectively. 

Judges, Race & Malpractice Settlements...

... though not all in one paper.

Henandez-Julian & Tomic have written How Elected Judges Respond to the Racial Composition of their Constituencies.  The Absract:

We take advantage of the variation in the judicial selection system - some judges are appointed, while others are elected - to address how the racial composition of a district affects sentencing. We find that appointed and elected judges act differently, suggesting that bias in the way that the laws are written cannot explain the entire difference in sentencing between blacks and whites. We find that blacks are more likely to be incarcerated, and receive longer sentences, but this effect diminishes as the percentage of the population that is black rises for a black defendant facing an elected judge. A one standard deviation increase in blacks' share of county population is enough to obliterate the difference in sentence length.

Philip Peters has written What we Know about Malpractice Settlements.  The Abstract:

The enclosed Article is the first comprehensive synthesis of two decades of empirical research on medical malpractice settlement. The portrait that emerges from this synthesis is both more reassuring and more complex than popular portrayals. Although the fit is not perfect, the merits generally drive the settlement process. Weak claims consistently fare the worst, toss-ups cases do better, and strong cases have the most success.

Do Attorneys Add Value?: Part II

I recently stumbled across yet another paper setting out to empirically assess whether attorneys generate independent value for their clients.  Put slightly differently--and in the words of the author, Erica Hashimoto (Georgia), the paper tests the long-held assumption that "there is no good reason for a [criminal] defendant to choose self-representation, and those who make that choice are either mentally ill or foolish."  Hashimoto's findings run against conventional wisdom.

Although in terms of research design and execution this paper is perhaps not as methodologically sophisticated as the Lederman tax court paper, Hashimoto's largely descriptive account is interesting and contributes to a foundation for future work.  (Orin Kerr provides additional commentary on the Hashimoto paper here.)

New Releases

Vining, Navarro, and Zorn:  Judicial Tenure on the U.S. Supreme Court, 1790-1868: Frustration, Resignation, and Expiration on the Bench.  The Abstract:

Though scholars have found that modern justices are frequently motivated to step down by political circumstances, we believe that their long-departed predecessors rarely had such a luxury. Specifically, we suggest that the focus on political factors often present in this literature is inappropriate in the period prior to the Judiciary Act of 1869 (16 Stat. 44). The absence of pensions and the rigors of circuit duty prevented early justices from reacting to political stimuli in the same manner as recent justices. In support of this proposition, we examine the effects of personal, institutional, and political factors on Supreme Court turnover, positing that departure behavior prior to the advent of judicial pensions was driven primarily by justices' personal and institutional, rather than political, circumstances.

Keith Hylton (BU): When Should a Case be Dismissed? The Economics of Pleading and  Summary Judgment Standards.  The Abstract:

This paper applies a simple economic framework to the choice between pleading and summary judgment as points at which a claim can be dismissed. It concludes generally that pleading standards should vary with the evidentiary demands of the associated legal standards and the social costs of litigation.  The common law's imposition of higher pleading standards for fraud claims is consistent with this proposition. The theory implies that the rigorous summary judgment standards that have been developed by antitrust courts should lead to a correspondingly rigorous assessment at the pleading stage.

Measures of Judicial Ideology

Both the existence and content of this blog show that empirical legal studies has made extraordinary progress in recent years in the field of applying empirical analysis to the effects of the law. However, the progress of empirical legal studies in analyzing legal institutions themselves -- courts, judges, and cases -- is more haphazard.

The central limitation is that the primary test of judicial ideology is inadequate. The traditional approach is to count the number of case outcomes categorized as liberal and contrast them to the number of cases labeled conservative. But case facts and the questions of law they present are not dichotomous; they represent the full breadth of legal inquiry. One case that is labeled liberal may be far more liberal than another case with the same label. Michael Bailey and Kelly Chang equated this mode of analysis to giving students a calculus exam and an addition quiz, and treating a score of 80% on each as equivalent.

APSA Law & Courts 2006 Article Prize

Congratulations to Lee Epstein (Wash U/Northwestern), Daniel Ho (Stanford), Gary King (Harvard), and Jeffrey Segal (SUNY-Stony Brook), whose article, The Supreme Court During Crisis (80 NYU L Rev 1 (2005)), won the APSA Law and Courts Section 2006 McGraw-Hill Award as "the best article published by political scientists on law and courts" during 2005.  Interestingly, as Jeff Yates (Georgia) notes, Epstein et al.'s awarding-winning political science paper was published in a student-edited law review.

A New Kind of Literature Review

At the recent American Law and Economics Association Annual Meeting held in Berkeley, John Donohue and Justin Wolfers presented a compelling analysis of the deterrence effect of the death penalty -- or rather, the lack of evidence thereof. Comparing execution rates with homicide rates, as well as using the natural experiment of the Furman abolition period, and contrasting U.S. trends to Canadian trends, Donohue and Wolfers cast considerable doubt on whether the death penalty has any deterrence effect. But Donohue and Wolfers look only at whether the death penalty deters crime, not whether the death penalty affects other criminal law matters, such as encouraging defendants in murder trials to accept plea bargains with harsher terms than they otherwise would, as Ilyana Kuziemko recently showed.

But more interesting than Donohue and Wolfers substantive case -- they ultimately conclude that it is entirely unclear whether the death penalty causes more or less murders -- is their methodology. They replicate the analyses of a handful of central studies that show there is a deterrence effect, and run a variety of robustness checks on each. As such, their paper provides a comprehensive re-examination of the primary data on the deterrence effect of the death penalty on crime.

Do Attorneys Add Value?: Evidence From Tax Court Litigation

A recent paper by Leandra Lederman (IU-Bloomington) and Warren Hrung (Fed. Reserve-NY) exploits unique aspects of the U.S. Tax Court to explore the possible effects of attorneys on case outcomes.  In the U.S. Tax Court, evidently, a substantial number of private litigants proceed pro se.  The Tax Court is also unique in that it retains records of settled cases (as well as, of course, tried cases).  Thus, armed with "a unique data set consisting of a random sample of settled and tried cases" the authors set out to empirically assess the influence of legal counsel on various case outcomes, including financial and disposition time (through settlement or trial).  Lederman and Hrung find, in part, that the presence of attorneys correlates with more favorable financial outcomes in tried but not settled cases.  The presence of attorneys does not influence disposition time.

Law firm starting bonus for Sup. Ct. Clerks hits $200K

The  Washington Post (via the WSJ Law Blog) is reporting that $200,000 is now the going rate for bonuses for Sup. Ct.  clerks.   Since most second-year BigLaw associates make $150K to $160K, a year at the Supreme Court is now a financial boon rather than a sacrifice:  $63,335 (what a SCOTUS clerk makes) + $200,000 = $263,335.   Is the value to the firm nonpecuniary or at least indirect--e.g., bragging rights on the number of former clerks?  Would a firm bill out a former Sup Ct clerk at a higher rate right out of the box?

It is not outside the realm of possibility that law schools, who also compete for former Sup Ct clerks, will eventually offer bonuses as well.   Granted, the external economies are different, and there is some evidence that a prestigious clerkship is not a valid predictor of the production and placement of future scholarship.  See Paul L. Caron & Rafael Gely, What Law Schools Can Learn from Billy Beane and the Oakland Athletics, 82 Tex L. Rev. 1483 (2004).   We'll see. 

Guest Blogger: Tonja Jacobi

It is with great pleasure that I introduce Tonja Jacobi as this week's guest blogger.  Tonja joined the Northwestern Law faculty in 2004, and earned her PhD in political science from Stanford University, where she wrote her dissertation on separation of powers constraints on the judiciary. She also holds a Masters from the University of California, Berkeley and a law degree from the Australian National University. Her areas of interest include judicial politics, game theory, American governmental institutions and constitutional law. Her current projects include a positive political theory of doctrine, an empirical study of the causes of litigiousness, a model of judicial signaling behavior, and a critical analysis of the use of state legislation as evidence of a national consensus in death penalty cases.

Luttig and the Distribution Tail of Lawyer Compensation

Judge Luttig's departure from the Fourth Circuit has renewed debate over judicial compensation.   Over at the W$J Law Blog, Peter Lattman cites Chief Justice Robert's 2005 statement that the low pay is a “direct threat to judicial independence" and asks whether federal judges need to make more money.  According to Ann Althouse, the clear subtext of Luttig's resignation letter is that "Federal judges are underpaid compared to their alternatives."

What does the data show?  I dug into my files to generate this chart, which appears to support the argument of lagging pay.   Obviously, there is a growing disparity between judge pay and jobs of comparable prestige in the private sector. (click to enlarge.)

Sources:  The American Lawyer, The Lawyer's Alamac

Yet, is corporate CLO and Am Law 50 partner the proper reference group?  Granted, Luttig is leaving for Boeing, and Chief Justice Roberts was a partner at Hogan & Hartson (Am Law #21 in 2003) before joining the D.C. Circuit.   But over the last twenty years, the earnings of the prototypical median law firm partner (25-29 years out of law school) have not grown nearly as fast.  Consider this chart:  (click to enlarge.)

Updated ICPSR Captial Punishment Dataset

ICPSR recently announced an update to its Capital Punishment in the United States, 1973-2004 dataset (ICPSR Study No. 4430).  (I've worked with prior versions.)  The dataset is put together and maintained by BJS (part of DoJ); ICPSR is the repository.  A summary of the dataset follows:

"This data collection provides annual data on prisoners under a sentence of death and prisoners whose offense sentences were commuted or vacated during the period 1973-2004. Information is supplied for basic sociodemographic characteristics such as age, sex, education, and state of incarceration. Criminal history data include prior felony convictions for criminal homicide and legal status at the time of the capital offense. Additional information is available for inmates removed from death row by year-end 2004 and for inmates who were executed."

In addition to the data and necessary documentation (e.g., codebook), ICPSR  also makes available ready-to-use SAS, SPSS, and Stata files.  Even those wholly uninterested in these data might wish to familiarize themselves with ICPSR's holdings.  ICPSR is a veritable treasure-trove for empiricists, including, increasingly, legal empiricists.

The Bar Exam as a measure of "competency" and "achievement"

Over at Concurring Opinions, Dan Solove discusses his recent satirical essay on the Multistate Bar Examination.  Dan’s other blog posts make clear that he favors the abolition of the bar exam. “In lieu of the Bar,” Dan writes, “states should permit all students who graduate from an accredited law school to become members of the Bar after working a certain number of supervised pro bono hours.”

I am somewhat skeptical of this proposal because of the agency problems that plague legal education. What law professors find interesting (and thus worth teaching) may not be a good preparation for a career as a practicing lawyer. The bar exam is at least a reference point for core substantive knowledge.  If we think it is a poor gatekeeping device, then the solution is professional engagement with state bar examiners … But most legal academics are rewarded for scholarship rather than teaching or service to the profession.  So we are back to incentives and agency problems; we complain but nothing changes. Sustained complaining is what is required, and the opportunity cost is quite high.

That said, the bar exam itself rests on some rather extravagant theoretical and empirical claims. For example, in a 1976 monograph entitled “The Relationships among Law School Predictors, Law School Performance, and Bar Examination Results,” the authors (two psychometricians from ETS) described the bar examination as “the ultimate test of competency” (p. 1). The authors also observed, “The bar examination measures, both essay and MBE, are in effect achievement tests of some the skills and knowledge developed, and tested, in law school” (p. 5, emphasis added). 

Really? Few law professors (or practicing attorneys, for that matter) would sign off on these statements.   Perhaps this statement is more accurate:  "The bar exam and the MBE are flawed, imprecise measures of competency and achievement; but they have persisted for decades with little or no change because of institutional incentives among legal educators."   This situation can be fixed, but who is going to bear the cost?
 

The Law Review Experiment

Should Eugene Volokh have received IRB approval before engaged in his law review experiment?  Dan Filler asks this question here.  As an experiment, the UCLA law professor participated in the UCLA Law Review write-on competition.

Luttig's Resignation

Judge Luttig’s letter of resignation (“effective immediately”) can be found on Boeing’s website here (via WSJ Law Blog).

Studying the Path of Precedent

Lindquist and Cross have written Empirically Testing Dworkin's Chain Novel Theory: Studying the Path of Precedent.  The Abstract:

This article provides an empirical study of the effect of precedent on judicial decisionmaking. Precedent is much analyzed as a controlling factor for judicial decisions but rarely tested for its effect. Some have argued that reliance on precedent creates a system of path dependence of the law, which may yield undesirable "herd" behavior. As a framework for the study of precedent, we use Dworkin's metaphor of a "chain novel," in which each deciding judge writes upon a background to which he or she must adhere, thereby influencing his decisions. This yields testable hypotheses about judicial decisionmaking. Given the considerable data on the ideological component of judicial decisionmaking, it provides a means to study the effect of precedent, as a constraint on judicial ideology. First, we examine "cases of first impression," for which there is no controlling precedent, like the very first chapter of a chain novel. The hypothesis is that the absence of precedent in these cases frees judges to use their ideology to a greater degree than in other cases with precedents, and this hypothesis is confirmed.  Judges render more ideological opinions in cases of first impression. The second study considers the growth of precedent over time in the context of 42 U.S.C. Section 1983 litigation.

The hypothesis is that as precedents accumulate, they steadily narrow the range of judicial discretion, so the effect of ideology should dwindle over time, as might be expected from a chain novel. The data falsify this hypothesis, as ideology seems to increase somewhat as a determinant as more precedents build up. The results suggest that precedent acts as only a limited constraint on judicial decisionmaking and the path dependence of a system of stare decisis is exaggerated.

Empirical Methodology as a Distinct Subfield in Law

In his comments about the distinct shift toward empirical papers at ALEA (a point echoed by many, including Bill Henderson), PrawfsBlawg's Ethan Leib brushes up against an equally interesting point (here as well): whether legal scholars should recognize empirical methodology as a distinct subfield.

The point warrants careful consideration.  Although the main thrust of Leib's comments is to suggest that empirical legal methodologists "get a free pass at needing to have a substantive field of expertise in the law," ironically, Leib's post fails to consider that methodology may constitute its own substantive subfield within law.  Methodologists in fields outside of law argue that it should.  (E.g., political scientists Gary King (Harvard) and Lee Epstein (Northwestern) wrote in 2002: "The law is important enough to have a subfield devoted to methodological concerns, as does almost every other discipline that conducts empirical research.")  Finally, to the extent that legal scholarship continues in its trajectory toward the empirical, it is particularly important that folks with substantive legal backgrounds and expertise contribute to and help shape the development of a subfield devoted to empirical methodology and technique.  As King & Epstein also noted: "Scholars toiling in the social, natural, and physical sciences can help, but a whole field [law] cannot count on others with differing goals and perspectives to solve all of the problems that law professors [legal scholars] may face."

Tufte Course on Presenting Data and Information

I first heard today about Edward Tufte’s course, “Presenting Data and Information.” Tufte is Professor Emeritus of Political Science, Statistics, and Computer Science at Yale and author of several books about presenting information, including Visual Explanations, Envisioning Information, and The Visual Display of Quantitative Information. His one-day course travels around the country, with upcoming sessions in Virginia, Oregon, Washington, Colorado, Pennsylvania, New York, Connecticut, and California. More information can be found here.

Anyone who has taken this course is certainly invited to share any comments about it.

An Empirical Analysis of Strict Scrutiny & Securities Fraud Class Actions

In public law, Adam Winkler (UCLA) has written Fatal in Theory and Strict in Fact: An Empirical Analysis of  Strict Scrutiny in the Federal Courts.

In private law, Randall Thomas (Vanderbilt) and James Cox (Duke) have written An Empirical Analysis of Institutional Investors' Impact as  Lead Plaintiffs in Securities Fraud Class Actions.

The Abstracts are below the fold.

Empirical Papers at ALEA

Over at Conglomerate, Fred Tung reports his impression of a strong empirical trend at this year's ALEA meeting.   I agree.   Every panel I attended had between one and three empirical papers.  From my perspective, there was some very high-end applied econometrics on display.  The phrase "robustness check" seemed to echo through the halls of the UC Clark Kerr Campus.   

Here are some of the empirical papers that I particularly enjoyed (in the order they were presented):

Statistical Significance: Judgment + Math

A recent discussion on the Statistical Modeling, Causal Inference, and Social Science Blog (SMCISS Blog?) provides an interesting discussion of some issues associated with tests of statistical significance. The discussion was motivated by a hypothetical situation involving a direct mail marketing campaign, the text of which I have slightly edited:

Group 1: 50,000 customers are mailed a catalog. Of these, 100 made a purchase, and the mean of their spending was $50 with a standard deviation of $10.

Group 2: 50,000 customers are mailed a catalog. Of these, 120 made a purchase, and the mean of their spending was $55 with a standard deviation of $11.

Some questions: Is there a statistically significant difference in the mean purchases associated with these two groups of customers? What is the appropriate N to use in testing for a difference? (If it’s 50,000, is there any reason to even bother with the test?) More importantly, for a decision-maker in this hypothetical company, what turns on whether the difference is statistically significant? These questions are discussed here and then here.

It’s reassuring to find people who are much wiser in the ways of quantitative methods struggling over “basic” questions—primarily, the important question of the value of hypothesis testing in this situation.

 

 

 

Stata: The Harry Potter Edition

Blogger, sociology professor, and author of Regression Models for Categorical Outcomes Using Stata, Jeremy Freese posted the following on his blog:

"On an entirely different tech front, I showed a friend of mine how she could easily realize her vision of creating Stata-HP, a version of Stata where various commands would be renamed to correspond to spells from Harry Potter. The "quietly" command, for instance, would be reimplemented as "silencio." She had embarked on assembling a list of commands and their spellname equivalents, but I haven't heard back from her since. Maybe its release can be timed to coincide with the final book in the series, perhaps with a set of Quidditch statistics to demonstrate it on."

We anxiously await the release of Stata-HP.

Bad Data and the Federal Data Quality Act

Six years ago Congress passed the little-known Data Quality Objectives Act (DQOA).  According to the Newsletter I just received from the Maryland Environmental Law Program and an article in the Maryland Bar Bulletin, "[t]he DQOA is a two-sentence mandate snuck into the fiscal year 2001 federal appropriations bill as a rider. It simply requires the Office of Management and Budget (OMB) to adopt mandatory guidance requiring each federal agency to adopt formal guidelines to ensure that information 'disseminated' by that agency meets certain data-quality criteria for 'quality, objectivity, utility, and integrity'."  But it seems data quality is in the eye of the beholder.  Industries, at least in the environmental context, are challenging state data and even potentially inaccurate spreadsheets under federal law.

The article linked to above asks, "Should environmental regulations be based upon a solid scientific foundation and reliable data before they become final and have the 'force and effect' of law?"  This, however, begs all sorts of questions:  What data is good enough?  Should agencies be allowed to regulate in the face of uncertainty where the risk is unknown but the potential harms may be severe?  What processes create good data in administrative agencies?  OMB and EPA guidelines for ensuring quality data can be found here and here.

I'd be interested if anyone knows if the Data Quality Act has been litigated in other contexts besides environmental law.

Some Effects of State Judicial Election Structure

Although some studies explore possible systematic variation between elected and appointed judges, comparatively less work assesses possible differences flowing from the structure of state judicial elections.  With respect to the latter, frustrating such efforts is that most judicial election states elect their judges in a uniform manner.  A slight wrinkle exists in Kansas, however, where partisan elections select states judges in 14 districts and noncompetitive retention elections select judges in 17 districts.  A recent paper by Gordon (NYU) and Huber (Yale) exploits this unique judicial selection feature in Kansas and explores how it influences sentencing.  They find that Kansas judges in partisan election districts sentence more severely than their judicial counterparts in retention districts.  Additionally, they provide evidence that suggests this difference arises because of the incentives created by electoral competition, rather than the selection of inherently more punitive judges in competitive districts.

Judicial Citations to Foreign Courts

David Zaring (Washington & Lee) has written The Use of Foreign Decisions by Federal Courts: An Empirical  Analysis.   The Abstract:

This article adds an empirical perspective to the debate over the use of foreign authority by federal courts. It surveys sixty years of federal court practice in citing opinions from foreign high courts, through a citation count analysis. The data reveals that federal courts rarely cite to foreign decisions, they do so no more now than they did in the past, and on those few occasions where they do cite to foreign decisions, it's usually not to help them interpret domestic law. Instead the citation of foreign decisions is best understood as a relatively rare phenomenon of judicial dialogue in cases where international issues are squarely presented by the facts. The article examines those few cases where federal courts have cited foreign decisions in some detail, and briefly considers some implications of the limited use of foreign decisions by federal courts.