Empirical Legal Studies

An Argument for Bayesian Approaches

In a recent paper, Should Legal Empiricists Go Bayesian?, Jeff Strnad (Stanford) makes a case for Bayesian (rather than frequentist) approaches that should interest Bayesians as well as non-Bayesians.  Jeff makes the particular point that Bayesian models can "enable a much more natural connection between the normative or positive issues that typically motivate such studies and the empirical results." He concludes that Bayesian methods "have much to offer legal empiricists." Despite the paper's "heft" (108 manu pp.--you've been forewarned), it is well worth a read.

Empirical Work on Property Issues (Blumenthal Guest)

The variety of topics to which ELS can profitably be applied--as shown, for instance, by the recent AALS conference and by the successful ELS conferences--is, of course, both encouraging and daunting.  As one perhaps atypical example, a month or so ago there was an interesting exchange on the PropertyProfs listserv, looking for examples of empirical legal articles examining property issues.  I am certain there are more than the ones identified, as this request focused on real property--and I hope readers might identify more, please!--but the ones that were mentioned are excellent examples of the sort of useful ELS work that can be done.  Thus, in no particular order, here are the ones mentioned:

• Joyce Palomar, The War Between Attorneys and Lay Conveyancers--Empirical Evidence Says "Cease Fire!", 31 Conn. L. Rev. 423 (1999)
• N. William Hines, Real Property Joint Tenancies: Law, Fact, and Fancy, 51 Iowa L. Rev. 582 (1966)
• N. William Hines, Personal Property Joint Tenancies: More Law, Fact, and Fancy, 54 Minn. L. Rev. 509 (1970)
• Michael H. Schill et al., The Condominium v. Cooperative Puzzle: An Empirical Analysis of Housing in New York City, available at SSRN: http://ssrn.com/abstract=502362
• Vicki Been & Ioan Voicu, The Effect of Community Gardens on Neighboring Property Values, available at SSRN: http:ssrn.com/abstract=889113
• Dean Lueck & Jeffrey A. Michael, Preemptive Habitat Destruction Under the Endangered Species Act, 46 J. L. & Econ. 27 (2003)
• Nicole Stelle Garnett, The Neglected Political Economy of Eminent Domain, 105 Mich. L. Rev. 101 (2006)
• Ann E. Carlson & Daniel Pollak, Takings on the Ground: How the Supreme Court's Takings Jurisprudence Affects Local Land Use Decisions, 35 U.C. Davis L. Rev. (2001)
• Leigh L. Linden & Jonah E. Rockoff, There Goes the Neighborhood? Estimates of the Impact of Crime Risk on Property Values from Megan's Laws, NBER Working Paper No.W12253 (2006)
• Robert Ellickson, Order Without Law (1991)
• Ward Farnsworth, Do Parties to Nuisance Cases Bargain After Judgment?: A Glimpse Inside the Cathedral, 66 Chi. L. Rev. 373 (1999)
• Mark D. West, Losers: Recovering Lost Property in Japan and the United States, 37 Law & Soc. Rev. 369 (2003)
• Robert H. Sitkoff & Max M. Schanzenbach, Jurisdictional Competition for Trust Funds: An Empirical Analysis of Perpetuities and Taxes, 115 Yale L.J. 356 (2005)
• Max M. Schanzenbach & Robert H. Sitkoff, Perpetuities or Taxes? Explaining the Rise of the Perpetual Trust, 27 Cardozo L. Rev. (2006)
• Lior Strahilevitz, How Changes in Property Regimes Influence Social Norms: Commodifying California's Carpool Lanes, 75 Ind. L.J. 1231 (2000)
• Lior Strahilevitz, Exclusionary Amenities in Residential Communities, 92 Va. L. Rev. 437 (2006).

Four Cheers for Meta-Analysis (Blumenthal)

One wonderful thing about this blog is the opportunity to learn about various different methodological approaches available for data analysis.  The workshops on methodologies and statistics (e.g. Northwestern's) are enormously helpful, and references for teaching and catching up on statistics have been as well.  Experimental work is very important, and relatively uncommon approaches such as MDS, network analysis, propensity score analysis, and others can give good insight into what is going on with both conventional and unconventional data.

But I'd like to highlight one approach that cuts across many of these, and in my view is one of the most important contributions ELS can make.  I'm a big fan of meta-analysis--of the quantitative synthesis of existing research studies--and advocate its use by ELS scholars, courts, agencies, practitioners, ice cream truck vendors, etc.

I've made some of my case for meta-analysis in a piece coming out in Temple Law Review, so I will be brief here.  First, descriptively, meta-analysis is just that--analysis one level up.  That is, primary research uses individual units (person, cases, courts, etc.) as the unit of analysis.  The individual units that meta-analysis involves, though, are the empirical studies themselves from a particular body of research.  Synthesizing the results of each study, the goals of meta-analysis are (1)  to identify the presence or absence of an effect in an existing empirical literature; (2) to evaluate the strength of that effect, for instance by summarizing the average effect across a set or subset of studies; and (3) to identify moderator variables, elements of the various studies that might have reliably affected their outcomes.  Thus, a good meta-analysis will identify and synthesize every study in a research area; find the average effect across all those studies in order to summarize the most current state of knowledge; and then systematically compare and contrast across studies, in order to identify methodological and substantive aspects of the various studies in a discipline that might be associated with the effect sizes they report.

Meta-analysis has a number of benefits, especially relative to the traditional narrative lit review: it is more comprehensive; it better avoids subjective judgments about what to include in a review; it is better at identifying "small" effects; it avoids over-emphasis on misleading statistical significance and p-values (focusing on effect sizes instead); emphasizes moderator variables; and, in my view, gives a better grounding for policy inferences.

Meta-analyses, unfortunately, are still rare in law reviews (Chris Guthrie and Dan Orr had one recently, but few others appear; they are cited sometimes, though).  They are used in court, usually in mass tort cases, but are often misunderstood by courts and experts.  My hope is that use of meta-analyses in ELS will increase--they serve well to summarize a body of research; they allow (or force) researchers and policy-makers to speak a "common language"--by making cherry-picking from existing studies more difficult--and, identifying moderator variables, they can prompt further research questions.

Written Opinions and Judicial Decisionmaking

In a very interesting new article forthcoming in Georgetown Law Review, Chad M. Oldfather of Marquette University Law School discusses whether writing really or necessarily makes judicial decisionmaking better, in light of psychology research and experimentation suggesting that under certain conditions a phenomenon of verbal overshadowing makes decisionmaking worse.  This article is an interesting companion to David Hoffman, et al.'s recent scholarship, discussed here , on when and why judges write opinions.

The abstract of Professor Oldfather's article follows:

Prior commentators, including many judges, have observed that writing provides an important discipline on the judicial decisionmaking process. Those commentators have uniformly assumed that the effect will always be positive - that is, that a decision rendered pursuant to a process that includes a written justification will always be better (however better is to be measured) than a decision unaccompanied by writing. According to this view, we should always, all things being equal, prefer a decision accompanied by an opinion to one without. All things are not equal, of course, and there are many situations in which the costs of generating an opinion uncontestably outweigh the benefits - such as in the case of evidentiary rulings made during the course of trial. Still, the understanding remains that writing will result in some positive contribution to the process.

This article calls that assumption into question. Drawing upon an emergent body of psychological research into the effects of both oral and written verbalization on decisionmaking effectiveness, it argues that certain types of decisions are likely to be worse if made via a process that incorporates writing. Decisions involving complex, context-intensive judgments that are best resolved via the weighing of largely inarticulable considerations are susceptible to a phenomenon called verbal overshadowing. In these situations attempts to justify a decision can lead the decisionmaker to focus on more readily verbalizable features of  the problem to the exclusion of those inputs that are more important to proper analysis.

The article also investigates the significance of writing to the fulfillment of the other two (aside from accuracy-enhancement)primary functions of judicial opinions, namely the creation and memorialization of precedent and the enhancement of legitimacy, and to consider the differing ways in which these functions are implicated at the trial and appellate levels. The goal is not so much to generate definitive answers as to better identify the costs and benefits provided by written opinions so as to more completely ground ongoing debates concerning when opinions should be issued, what form they should take, and who should author them.

What Determines Traffic Citations?

It's all about safety, right? Er, maybe not. Michael Makowky and Thomas Stratmann of George Mason offer this analysis in a recent paper:

In this paper we study the political economy determinants of traffic fines. Speeding tickets are not only determined by the speed of the offender, but by incentives faced by police officers and their vote maximizing principals. Our model predicts that police officers issue higher fines when drivers have a higher opportunity cost of contesting a ticket, and when drivers do not reside in the community where they are stopped. The model also predicts that local officers are more likely to issue a ticket when legal limits prevent the local government from increasing revenues though other instruments such as property taxes. We find support for the hypotheses. The farther the residence of a driver from the municipality where the ticket could be contested, the higher is the likelihood of a speeding fine, and the larger the amount of the fine. The probability of a fine issued by a local officer is higher in towns when constraints on increasing property taxes are binding, the property tax base is lower, and the town is less dependent on revenues from tourism. For state troopers, who are not employed by the local, but the state government, we do not find evidence that the likelihood traffic fines varies with town characteristics. Finally, personal characteristics, such as gender and race are among the determinants of traffic fines.

The paper is available on SSRN here and discussed in the Chicago Tribune here. Various accounts of the importance of considering monetary incentives when evaluating traffic enforcement patterns can be found here, here, here, here, here, here, here, here, etc. (H/T to theNewspaper.com for the links in the last sentence.)

Order of Names in Co-Authored Scholarship

According to this recently posted paper, the order of your name in a co-authored article may have an effect on your career.  Here is the abstract for Mirjam Van Praag & B.M.S. Van Praag, "The Benefits of Being Economics Professor A (and Not Z)":

Alphabetic name ordering on multi-authored academic papers, which is the convention in the economics discipline and various other disciplines, is to the advantage of people whose last name initials are placed early in the alphabet. As it turns out, Professor A, who has been a first author more often than Professor Z, will have published more articles and experienced a faster growth rate over the course of her career as a result of reputation and visibility. Moreover, authors know that name ordering matters and indeed take ordering seriously: Several characteristics of an author group composition determine the decision to deviate from the default alphabetic name order to a significant extent.

(Many thanks to Danny Sokol for the tip.)  Perhaps Mirjam and B.M.S. are married; because of the ordering of the names, I would presume that Mirjam did more work on the article.

Study of N.B.A. Sees Racial Bias in Calling Fouls

While Michael linked to the paper below, it is also the subject of a NY Times article entitled Study of N.B.A. Sees Racial Bias in Calling Fouls.  Two highlights from the article:  First, the caption under the headlining photo--"Minnesota Timberwolves guard Mike James, left, said he did not think he was treated differently by white and black officials"--says a lot about the need for systematic empirical inquiry.  Second, the NY Times engaged in peer-review--"Three independent experts asked by The Times to examine the Wolfers-Price paper and materials released by the N.B.A. said they considered the Wolfers-Price argument far more sound."  The article also discusses multivariate regression and  Dallas Mavericks owner Mark Cuban's viewpoint on statistics.  It's a must read!!!

Vault Top 25 Underrated Schools: What Does It Mean?

Vault, Inc, which is a publisher that collects industry information on various professions, recently released its listing of the Top 25 Most Underrated Law Schools.   (Hat tip: Paul Caron & Volokh.)  The findings are derived from a survey of  512 law firm recruiting managers, hiring partners, and corporate counsel, who were asked "to name law schools that, based on their experience as hiring managers, are underrated."

As I read the list, I wondered, what is the practical implication of a law school being underrated by the people who make hiring decisions for entry level lawyers?  Presumably, it means that graduates of certain law schools tend to perform better than their school's U.S. News ranking would suggest; thus, legal employers are more likely to hire them. 

If this is true, what is the source of superior performance? Here are two possibilities:

1. Stronger Students.   Some schools may enroll a stronger student body than their rank might suggest.
2. Better Education.  Some law schools may equip graduates with more or better skills than other schools of comparable rank

Based on some preliminary statistical analysis, there is fairly clear quantitative evidence for the first hypothesis.  There is also some qualitative evidence for the second--enough to warrant some additional research. 

These research questions are really important to the ranking debate because their answers could reveal a mechanism by which some students will discount USN.  If an "underrated" school offers a better entree to coveted legal employers, why go to the higher ranked school, especially if the underrated school costs significantly less?  Law schools that understand these dynamics are in a better overall competitive position.

After the jump, I elaborate on these findings, including methodology.

A Serious Debate on Empiricism and Theory in Bankruptcy

Last April, we posted the abstract of Robert Rasmussen's (Vanderbilt) working paper, "Empirically Bankrupt," which critiques three empirical papers in the bankruptcy area.  The abstract concluded that "[f]or empirical work to be credited, at a minimum, it has to look in the right place, ask the right question and draw the right inferences. When empirical work fails to cross this threshold, it conclusions must be rejected."

One of the articles being critiqued was Elizabeth Warren's (Harvard) & Jay Lawrence Westbrook's (Texas) recent study, Contracting Out of Bankruptcy: An Empirical Intervention, 118 Harv. L. Rev. 1197 (2005).  A few months ago, Warren & Westbrook posted a rejoinder, "The Dialogue Between Theoretical and Empirical Scholarship," that not only addresses Rasmussen's criticism, but also tries to articulate an optimal relationship between theory and empiricism.   The entire exchange between Rasmussen and Warren & Westbrook is worth reading. Here is the abstract of the rejoinder essay:

In this essay we offer brief reflections on the best process for critiquing empirical work in law and sustaining an engagement between theoretical and empirical approaches. We emphasize the importance of theoretical work in helping to shape the scholarly agenda, but we urge that theory should be more closely tied to fact. We illustrate our argument by responding to a recent critique of our own empirical work by Professor Rasmussen. His principal claim is that our work should be discounted because we reported on all business bankruptcies, both those of entrepreneurs and those in corporate form. In response, we reanalyze our data, separating the individuals from the corporations; in every case the re-analyzed data support the conclusions of our original paper to the same extent or more strongly.  Similarly, his other claims about our work are shown to be incorrect.

Judges Use Law Reviews "Like Drunkards Use Lampposts: More For Support Than for Illumination"

Paul Caron  links to a devastating (and devastatingly funny) article in the New York Lawyer, entitled "Federal Judges Discuss Law Reviews Usefulness."   Paul excerpted some the sharpest lines:

Kicking off a recent friendly-but-frank discussion about the relevance of contemporary law review articles, Dean David Rudenstine of the Benjamin N. Cardozo School of Law offered one description: "useless blather puffed up with self-indulgence" ...

Judge Sack returned to the not-so-funny problem under consideration: today's highly theoretical articles are largely ignored and seldom cited by judges, a dramatic turnabout from a generation ago when the mostly practical content of law reviews was a significant element of judicial decision-making. Judge Sack was sorry to say that the bench now uses law reviews "like drunkards use lampposts, more for support than for illumination." ...

"If the academy wants to change the world, it must decide if it wants to be a part of the world." ...

Although Judge Sotomayor agreed that brainy law professors should not trouble themselves by contemplating reactions from the bench to their writings, she leveled a sharp gaze at the Cardozo Law faculty and declared, "If you think that judges are not as capable of creative thought as you are, I beg to differ." She added, "My question to academics: do you really think you're serving some function to someone?"

Wow, that is some really straight talk.  For the record, law reviews had a sketchy record 40 years ago, see Lowell J. Noteboom & Timothy B. Walker, The Law Review–Is it meeting the needs of the Legal Community?, 44 Denver L. J. 426 (1967).