Empirical Legal Studies

"Deal or No Deal": NYT Story on Forthcoming JELS Article re: Case Settlement Decisions

Last Friday's NYT (8.8.08) included a lead story in the Business section (at C1) on a forthcoming JELS article. The subject of the NYT piece--a forthcoming article in 5:3 JELS (Sept. 2008)--reports results from a study of the financial consequences of case settlement decisions. To better assess the financial cost of going to trial, the study analyzes cases in which a settlement offer was considered, but rejected in favor of proceeding to either arbitration or trial. The findings reveal the influence of contingency fee arrangements and the availability of insurance coverage on plaintiff and defendant settlement decisions, respectively.

Advantage Employers: Reversal Rates in Employment Disputes

An article recently posted on SSRN provides some interesting data about how employers and employees fare when arbitrators’ decisions are reviewed in court.  In Do Courts Create Moral Hazard? When Judges Nullify Employer Liability in Arbitration: An Empirical Analysis, Michael Leroy argues that the possibility of such review – especially when the arbitration clause provides for de novo review, as many do – creates a systematic advantage for employers.  Leroy documents a growing number of bases on which courts (particularly state courts) vacate arbitration awards, providing more opportunities for successful challenges to arbitrators’ decisions.  Perhaps most importantly, however, Leroy measures the rate of reversal of arbitrators’ decisions.  Out of a dataset of 267 separate arbitration decisions, Leroy found that federal courts are routinely extremely deferential to arbitrators’ decisions, upholding decisions for both employers and employees at similar and extremely high rates.  As a general matter federal courts upheld awards for employees at a rate of 85% and for employers at about 92%.  In state courts, however, the picture is more complex.  There were larger differences between trial courts and appellate courts, for one thing, but more striking is the difference in upholding the awards for employers as opposed to awards for employees, particularly at the appellate level.  State trial courts and appellate courts both upheld awards in favor of employers at a rate of about 87%.  But for awards for employees, trial courts upheld them 77.6% of the time, while appellate courts upheld only 56.4% of such awards.  The significantly higher rates of vacatur of employee awards in state courts, Leroy argues, creates a moral hazard for employers.  Their incentives are to require employees to sign arbitration agreements that allow for expansive review in state court.  If the employer wins in the arbitration, its chances of prevailing under court review remain quite high.  On the other hand, if the employer loses in the arbitration, the generous review offered by state courts essentially gives it a second bite at the apple.  As a result, Leroy argues, employers may have less incentive to curtail legally risky behavior because they are less likely to have to pay for the consequences if sued.
This article, while quite different in its focus, is reminiscent of the findings of two articles examining differential appellate court treatment of plaintiffs versus defendants.  In Plaintiphobia in the Appellate Courts: Civil Rights Really Do Differ From Negotiable Instruments,  a 2oo2 article in the University of Illinois Law Review, Theodore Eisenberg and Kevin Clermont found that in federal civil rights employment cases that terminated between 1988 and 1997, defendants who appealed trial losses prevailed on appeal 44% of the time. In other words, where a defendant appealed a verdict, generally entitled to enormous deference, there was an almost even chance that the appellate court would reverse. In contrast, an employment plaintiff who appealed from a pro-defendant verdict had only a 6% chance of prevailing. As a point of comparison, the overall reversal rate from all civil trials was 18%.
In a more recent follow-up (blogged about here), Plaintiphobia in State Courts? An Empirical Study of State Court Trials on Appeal by Theodore Eisenberg and Michael Heise, examined the outcomes of more than 8000 trials and about 550 appeals from 46 large counties.  They found that in general, plaintiffs fare worse on appeal than defendants and that the appellate courts are more deferential to bench verdicts than jury verdicts.  Consistent with the first Plaintiphobia article and with Leroy’s findings, the plaintiff/defendant disparity was very stark in the context of employment cases – with 61.5% of verdicts for plaintiffs reversed and 38.5% of the verdicts for defendants reversed.  (Both of these reversal rates are higher than the overall numbers across all case types – 41.5% of verdicts for plaintiffs reversed and 21.5% of verdicts for defendants.)
All three articles discuss possible reasons for the observed disparities between plaintiffs and defendants.  The Plaintiphobia articles do not find strong evidence to support selection effects, and conclude that their findings are consistent with attitudinal effects – specifically that appellate judges believe (possibly erroneously) that juries are biased towards plaintiffs.  Leroy attributes the disparities at least in part to the expansion of bases for reversal of an arbitration award – a doctrinal development.  Moreover, there is the possibility of a snowballing effect on doctrine – the more pro-employer cases that are decided, the more pro-employer the law becomes.  At minimum, however, these articles collectively raise questions about whether the appellate playing field is level for employers and employees.

Update: It's worth noting that the two Plaintiphobia papers analyze appeals in all different kinds of cases, not limited to employment cases.

How the "Cravath System" Created the Bi-Modal Distribution

The bi-modal distribution (graphic to the right) continues to generate interest in the blogosphere.  See, e.g., Greg Mankiw, Right Coast, Broken Symmetry.   The  chart summarizes the starting salaries for lawyers who graduated from law school in 2006.  One reason the bi-modal structure is so jarring is that it demonstrates that measures of central tendency, such as average or median, are not necessarily reliable guides for law students' future earning power.  In conventional labor markets, that disconnect is rare.

NALP recently dug into its archives to determine whether this stratification is a persistent feature of the entry level law market.  See NALP Bulletin (Jan. 2008).  It turns out that 15 years ago, the market followed a much more traditional distribution.   The chart below summarizes the salaries for the class of 1991.

The 1991 graph is right skewed but bears some resemblance to a normal curve.  Below is the graph for 1996:

The rightward skew is a bit more pronounced and the area under the $75K to $85K range is becoming more substantial.  A more seismic shift is seen in 2000 (below) with the emergence of a second mode at the $125K price point. 

At the height of the Internet boom, a remarkable 14% of all entry level lawyers took jobs at the $125K level.  According to NALP, "never before had a single salary so dominated the landscape."  Under the 2006 bimodal distribution (see chart above), 44% of graduates received entry-level salaries in the $40K to $60K range; yet, the second mode moved further to the right ($135K to $145K) and grew to 17% of all graduates.

What are the market forces that have created this peculiar salary structure?  In my working paper, "Are We Selling Results or Résumés?: The Underexplored Linkage Between Human Resource Systems and Firm-Specific Capital," I posit that the runaway $160K mode is a confluence of two factors:  (1) the continued growth in the corporate legal services market, primarily due to the growing scale and scope of transnational corporate activity; and (2) law firms' nearly universal adherence to the "Cravath system," which purports to hire the best graduates from the best law schools and provide them with the best training.  More after the jump. ...

Network Analysis of Law Clerks and Judges

Over the next couple of weeks I plan to blog about some new research, with a particular focus on unusual techniques or interesting research questions.  Several of the research projects I will discuss came to my attention through the Law and Society Conference in Montreal at the end of May.

One interesting presentation was a paper entitled Hustle and Flow: A Social Network Analysis of the American Federal Judiciary by Daniel M. Katz and Derek K. Stafford, both political science graduate students at the University of Michigan.   Katz and Stafford are interested in the social structure of the judiciary and, more controversially, whether that structure has an effect on doctrine or case outcomes.  Different from the prevailing models of judicial behavior (attituindalism, legalism, or the strategic model), their hypothesis is that judges -- at least sometimes -- are influenced by "peer effects," not just by their own political views or by legal sources.  (Such social pressures could be a partial explanation for the panel and circuit effects documented on appellate courts.  See, for example, Kastellec, Jonathan P., "Panel Composition and Voting on the U.S. Courts of Appeals Over Time" (May 14, 2008). Available at SSRN: http://ssrn.com/abstract=1012111) and Kim, Pauline T., "Deliberation and Strategy on the United States Courts of Appeals: An Empirical Exploration of Panel Effects" (March 31, 2008). Available at SSRN: http://ssrn.com/abstract=1115357 ).

In order to study these social or peer effects, they must find a way to describe the social networks of the judiciary.  As an initial attempt to do so, Katz and Stafford employ new network analysis techniques to measure the paths that law clerks take between judges.  In other words, they chart the likelihood that a law clerk for, say, Judge Kozinski, will later clerk for Justice Kennedy, or that a law clerk for a particular district court judge will go on to clerk for a particular appellate court judge.  Their study "visualizes law clerk traffic" during the last Rehnquist natural court and produces some interesting representations of the relationships between judges.  (They identified about 900 clerks who moved from one judge to another during this period.)

The graphic depiction of these law clerk moves is one of the more interesting aspects of the study.  In one depiction, Supreme Court justices are, not surprisingly, clustered in the middle, but -- more surprisingly -- district court judges are "suffused throughout the network," not relegated to the periphery.  This is one representation, Katz and Stafford suggest, of the fact that judges with equivalent institutional authority in fact have different levels of influence.  Below is one such figure from the paper -- the Kamada Kawai Energized Network.  (Yellow nodes are Supreme Court justices; green are appellate court; and blue are district court judges.)

 

 

It will be interesting to hear more from these authors as they apply their methods to other aspects of the judiciary.

The Geography of Stock Prices

There is an interesting story in the Sunday New York Times about a recent study by George Korniotis (Fed Bd of Governors) and Alok Kumar (Texas Business), entitled "Long Georgia, Short Colorado: The Geography of Return Predictability" (on SSRN here). 

In a nutshell, the article documents that individual investors are overinvested in businesses that are located closer to them. Thus, when state or regional economic conditions deteriorate relative to the country as a whole, the sale of stocks--to maintain one's standard of living--has a predictable, disproportionate effect on the stock prices of companies headquartered nearby. There is also some interesting methodology in this paper, at least for those of us interested in testing the effects of geography on business or law.  Here is article abstract:

This paper shows that returns of U.S. state portfolios are predictable. In the presence of local bias and incomplete risk sharing, consumption smoothing motives of local investors generate predictable patterns in the returns of local stocks. Specifically, local investors require higher average future returns to hold risky local stocks when local economic conditions worsen and they face stronger borrowing constraints. The state portfolio returns are predictable both in the short-run (one quarter ahead) and the long-run (up to 24 quarters ahead). The predictability is stronger among less visible firms and in regions where investors exhibit stronger local bias and hold more concentrated portfolios. Trading strategies that exploit the state-level predictability earn annual risk adjusted returns of around 6-8%. Overall, these results indicate that the stock return generating process contains a predictable local component.

Transfer Students -- The Data

Brian Leiter has just posted rankings of the Top 10 law schools by percentage of transfer students.  See Schools that Take the Largest Number of Transfers (Relative to the Size of Their 1L Class).  It is generating a lot of traffic, see, e.g, Paul Caron and Larry Ribstein, in part because Brian names names. 

Last fall I posted charts that showed the same data ordered by 2007 U.S. News rankings.  I took a lot of heat in the comments from readers who wanted to defend these liberal transfers--many by people who benefited from the ability to trade up.   This system of liberal transfers has other defenders as well.  For example, professors at higher ranked schools often justify these policies by arguing that they are opening doors for hardworking students who have proven themselves.  My critique, however, was based on a macro-level social wealth perspective--i.e.,  do the benefits of liberal transfers policies outweigh the costs?   Here are some facts:

More transfers.  Between 1997 and 2004, non-academic 1L attrition, which is the ABA-LSAC Official Guide category that would include transfers, has increased at statistically significant levels.  See Morriss & Henderson, Measuring Outcomes, Fig. 2 and accompanying text.  I would posit that this coincides with a rankings payoff of higher entering credentials, not a rethinking of admissions policies driven by principles of equity and opportunity.  We should not confuse colorable justifications, which lawyers are expert at, with underlying motivations of self-interest.

Rankings gains.  By the way, these gaming strategies actually produce higher LSAT scores relative to non-gaming schools.    See Henderson & Morriss, Student Quality as Measured by LSAT Scores, at 191. 

Effects within schools.  According to the 2005 LSSSE Annual Report, transfer students were less likely to:

• Perceive their relationships with other students to be as positive as students who did not transfer.
• Work with other students outside of class to complete an assignment.
• Have serious conversations with students who are different from themselves.
• Discuss ideas from reading or assignments with others outside of class.
• Work on a paper or project that required integrating ideas.
• Participate in cocurricular activities.

P. 14-15.  The report continues, "These findings underscore that many of the strongest student relationships are formed during the first year of law school before transfer students join the campus community."  (p. 15). Note these findings describe group results.  Please spare me anecdotes in the comments about well-adjusted transfer students.  We cannot generalize, and thus make policy, from individual data points.  We need a representative sample, which LSSSE provides.

Harms to Curricular Innovation.  If the best students from Tier 3 and 4 are siphoned by Tiers 1 and 2, lower ranked schools have one hell of a time demonstrating the value of their curricular innovations.  And these innovations often require immense commitments of time and money.  The substantive value may be there, but the school will not get an adequate return on the investment.   So incentives to innovate are diluted.

So my point is very simple:  Instead of shrinking the 1L class and taking more transfer students, a better system would admit more 1L students using whole person review (the policy that all of the legal academy allegedly embraced in the Grutter litigation) rather than a cynical numbers-driven approach designed to maximize entering credentials for U.S. News purposes.   I realize that some people would be made worse off under this system (and these people tend to know who they are, so they are noisy).  In contrast, the people who would benefit are largely nameless and invisible.  But this approach eliminates the educational costs of excessive transfers and preserves incentives for healthy institutional competition based on curriculum and teaching.  From a social wealth perspective, which is better?

At the end of the day, the increase in transfers represents a giant collective action problem.  But it could be solved if U.S. News reworked its methodology.   The putative regulator here is the ABA Section on Legal Education and Admission to the Bar.  Are they even paying attention?  I often wonder.  After the jump are the charts that show net transfers by U.S. News rank.  It is very ugly.

Behavioral Economics Primer

I recently ran across this audio file of a talk given at Harvard Law School in 1995 by Charles Munger, entitled "Causes of Human Misjudgment."   Munger is a Harvard Law alum and a founding partner of the L.A. law firm of Munger, Tolles & Olsen.  He later left the firm to run an investment fund.  In the mid-1970s, he joined Berkshire Hathaway to serve as Vice-Chairman with Warren Buffett.

Make no mistake:  Munger is explicitly talking about the intersection of economics and psychology, acknowledging that the nascent (at the time) field of behavioral economics was on the right path and citing insights of Daniel Kahneman and Amos Tversky (note seven years before Kahneman won the Nobel Prize).

I am particularly interested in applied behavioral economics.  And here I don't mean writing papers on the topic;  rather, I mean honing my own decisionmaking processes to eliminate bias and susceptibility to manipulation.  For this purpose, I doubt I will ever find a better resource.  A written version of this talk, including an interesting preface, is online here.   It has quite a few things to say about academia and the pervasive problem of the "truffle hound."  (You will have to read/listen to the links to figure out the truffle hound reference.) 

Finally, Munger's remarks inspired Paul Brest (Hewlett Foundation, former dean at Stanford) and Laura Hamilton Krieger (UC Berkeley) to create a law school  course on professional judgment and decisionmaking at Stanford Law.  The course began in the early 1990s and continues to this day (materials are online at www.professionaljudgment.org).   Several prominent behavioral law & econ scholars took this course while students at Stanford, including Russell Korobkin (UCLA), Jeff Rachlinski (Cornell), and Chris Gurthie (Vanderbilt).    In case there was any doubt, cornerstones of the course appear to be empiricism, applied probability theory, and due attention to disconfirmatory evidence in the face of cognitive biases.  Great stuff.

Science and the Law

For those interested in the continued (and sometimes difficult) relationship between science and the law, I encourage you to check out my Vermont Law School colleague Craig Pease's regular "Science and the Law" column in The Environmental Forum.  Check out the latest edition entitled The Absurdity of Individual Harm.

BJS Report on Felony Defendants

Another recent BJS report, Felony Defendants in Large Urban Counties, 2004, analyzes data collected from a representative sample of felony cases filed in the nation's 75 largest counties during May 2004. Murder cases were tracked for up to 2 years and all other cases for 1 year to provide an overview of the processing of felony defendants from case filing to disposition and sentencing. Data highlight the demographic characteristics of felony defendants and types of arrest charges. The report also includes in-depth information on the criminal record of felony defendants, including criminal justice status at the time of arrest and the number and type of prior arrests and convictions. It describes conditions of pretrial release (bail amounts, type of release bonds, and pretrial misconduct), adjudication outcomes (dismissal, diversion, guilty plea, trial conviction rates), and sentencing data for convicted felony defendants. Notable findings include:

• Two-thirds of felony defendants were charged with a drug or property offense.
• More than three-fourths of felony defendants had a prior arrest history, with 53% having at least five prior arrest charges.
• Fifty-seven percent of felony defendants received a pretrial release prior to adjudication.

Warren Buffett and Efficient Market Theory

I like Warren Buffett--or at least his public persona and mythology--for a lot of reasons.  He makes a lot of money without much use for computers or technology, he claims to invest only in what he understands (e.g., insurance, candy, etc.), he treats people with respect (including his shareholders), and he says that his taxes are too low.  Buffett also agreed to give away 85% of this wealth to the Bill & Melinda Gates Foundation upon his death.  He could have built a monument to himself, but instead he concluded that his money would go further by giving it to Gates. 

But my favor reason might be in the below abstract from a top-ranked paper on SSRN: that  Warren Buffett et al.'s 31 year track record is not consistent with Efficient Market Theory.    See Gerald S. Martin (American/Texas A&M) & John Puthenpurackal (UNLV), Imitation is the Sincerest Form of Flattery: Warren Buffett and Berkshire Hathaway:

We analyze Berkshire Hathaway's equity portfolio over the 1976 to 2006 period and explore potential explanations for its superior performance. Contrary to popular belief, we find Berkshire Hathaway invests primarily in large-cap growth rather than "value" stocks. Over the period the portfolio beat the benchmarks in 27 out of 31 years, on average exceeding the S&P 500 Index by 11.14%, the value-weighted index of all stocks by 10.92%, and a Fama and French characteristic-based portfolio by 8.56% per year. Although beating the market in all but four years can statistically happen due to chance, incorporating the magnitude by which the portfolio beats the market makes a luck explanation extremely unlikely even after taking into account ex-post selection bias. We find that Berkshire Hathaway's portfolio is concentrated in relatively few stocks with the top five holdings averaging 73% of the portfolio value. While increased volatility is normally associated with higher concentration we show the volatility of the portfolio is driven by large positive returns and not downside risk. The market appears to under-react to the news of a Berkshire Hathaway stock investment since a hypothetical portfolio that mimics the investments at the beginning of the following month after they are publicly disclosed also earns significantly positive abnormal returns of 10.75% over the S&P 500 Index. Our evidence suggests the Berkshire Hathaway triumvirates of Warren Buffett, Charles Munger, and Lou Simpson possess investment skill unlikely to be explained by Efficient Market Theory.

So here is my question:  If you know how to make money like Buffett et al., what would you do:  apply the strategy in the market and make supra normal returns, or publish it all in a finance journal to settle an academic debate (and ruining the strategy prospectively)?