Empirical Legal Studies

Judges Use Law Reviews "Like Drunkards Use Lampposts: More For Support Than for Illumination"

Paul Caron  links to a devastating (and devastatingly funny) article in the New York Lawyer, entitled "Federal Judges Discuss Law Reviews Usefulness."   Paul excerpted some the sharpest lines:

Kicking off a recent friendly-but-frank discussion about the relevance of contemporary law review articles, Dean David Rudenstine of the Benjamin N. Cardozo School of Law offered one description: "useless blather puffed up with self-indulgence" ...

Judge Sack returned to the not-so-funny problem under consideration: today's highly theoretical articles are largely ignored and seldom cited by judges, a dramatic turnabout from a generation ago when the mostly practical content of law reviews was a significant element of judicial decision-making. Judge Sack was sorry to say that the bench now uses law reviews "like drunkards use lampposts, more for support than for illumination." ...

"If the academy wants to change the world, it must decide if it wants to be a part of the world." ...

Although Judge Sotomayor agreed that brainy law professors should not trouble themselves by contemplating reactions from the bench to their writings, she leveled a sharp gaze at the Cardozo Law faculty and declared, "If you think that judges are not as capable of creative thought as you are, I beg to differ." She added, "My question to academics: do you really think you're serving some function to someone?"

Wow, that is some really straight talk.  For the record, law reviews had a sketchy record 40 years ago, see Lowell J. Noteboom & Timothy B. Walker, The Law Review–Is it meeting the needs of the Legal Community?, 44 Denver L. J. 426 (1967).

Co-Authorship

One article I have been meaning to blog about is this profile of economist Kevin Murphy in the Nov/Dec 2006 issue of University of Chicago Magazine.  Murphy has an impressive list of accomplishments, including the John Bates Clark Medal of the American Economic Association, which is given once every two years  for the most outstanding American economist under age 40 (1997), and a MacArthur Foundation "genius grant" (2005).

Murphy is quite a character, and the article provides some fascinating details on his career history.  But this passage on co-authorship really caught my eye:

"[Regarding co-authorship,] I think most people get a better product that way,” he explains. “You pool insights and eliminate oversights. First-rate coauthors make working on projects more fun and rewarding. Plus, writing is a pain.” Since earning his PhD and joining the faculty in 1986, he has written every one of his 60-plus published papers with a former teacher, a close colleague, or students. ...

As a shy but affable 25-year-old grad student, Murphy confessed to [his advisor] Topel that his goal at that stage in life was “to become the world’s best coauthor.” By the time he was 35, he was there.

Wow, that is quite a testament to collaboration.  To the right is a picture of Murphy, who is an avid woodworker, in his workshop at home.   Apparently, it is impossible to locate a picture of Murphy without his signature baseball cap. 

Relationship between Journal Prestige and Citations

Paul Caron has a post an interesting forthcoming study by Andrew Oswald (Warwick Economics) in Econometrica on the relationship between journal prestige and citations.  See Andrew Oswald, An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers.  Since a single body in the UK makes funding decisions for university research, there is significant pressure to rely upon measures of productivity that take into consideration the relative prestige of  faculty members' placements.  Because these journals are peer-reviewed, and the hierarchy of journal prestige in each discipline is fairly established, placement -- so the argument runs -- should be construed as a strong signal of quality.

Oswald's study is important, though the abstract is -- in my opinion -- quite misleading.  This claim caused me to the read the whole study: 

The paper finds that it is far better to publish the best article in an issue of a medium-quality journal like the Oxford Bulletin of Economics and Statistics [a middle-of-the-pack journal] than to publish the worst article (or often the worst 4 articles) in an issue of a top journal like the American Economic Review.

Oswald used citation counts from articles in six economics journals over a twenty-five year period to evaluate the reliability of placement as a quality signal.  But there is no implicit suggestion in the article that a scholar would be better off trading down.

Basically, it boils down to this: Placement is correlated with citation, but as a proxy for quality, there are large Type I and Type II errors.  Articles in AER and Econometrica (#1 and #2 in prestige) did indeed get more citations. Yet, roughly 16% of the articles in the less prestigious journals garnered more citations than the median AER or Econometrica articles.  The most cited articles in the less prestigious journals garnered 10 times or more the citations than the four least cited articles in each issue of AER or Econometrica.

In my opinion, here is the good news: 

Empirical Analysis of the Biglaw Tournament

Lately there has been a lot of buzz in the blogosphere on the associate-partner "pay gap" (Jeff Lipshaw, titled "The Days of Whine and Roses"), the application of Marxist economic theory to corporate law firms (David Luban), and what all the huge salaries and profits augur for the future of legal profession and legal education (Brian Tamanaha). 

All of it is worth reading, especially Tamanaha's insights on the growing fissure between elite and nonelite law schools.  (For a scholarly treatment of this topic, see Randolph Jonakait, The Two Hemispheres of Legal Education and the Rise and Fall of Local Law Schools, NYLS L Rev (forthcoming 2007)).

David Zaring and I recently posted our "Young Associates in Trouble" essay, which provides some new empirical evidence on the BigLaw tournament.   The essay reviews two recent novels that portray elite law firms (one in DC, the other in London) as an unending marathon of boring and substantively unfulfilling work.  In turn, we compare these fictitious accounts with findings from a unique dataset based on the AmLaw Midlevel Survey, which includes questionnaire responses from several thousand 3rd, 4th, and 5th year associates. 

So what is the key takeaway?  As shown in the table below, virtually every principle of enlightened management and social responsibility is either (a) associated with lower firm profits, or (b) irrelevant to the bottomline.

The essay also includes some multivariate regression models that reveal that firms actually make more money when associates work on less interesting work or report a greater likelihood of leaving the firm within the next two years.  In contrast, family friendliness, training and guidance, and feedback appear to have no effect on profits.

More empirics and another graph after the jump.

Empirical Study on Effects of Buckhannon

The economics of class action litigation is a fascinating topic.  This terrain shifted substantially in 2001 when the Supreme Court's decision in Buckhannon made it harder for lawyers to collect attorneys' fees under the civil rights fee-shifting statutes.   In essence, this decision limited fee-shifting to cases that result in a judicial judgment or consent decree.  Prior to Buckhannon, virtually every circuit had adopted the "catalyst theory", which permitted recovery when the lawsuit in question had produced some or all of the relief requested by the plaintiff class.

Without this potential pool of money available to pay for legal services, we might expect the number of civil rights class actions to taper off.  At the time, the Supreme Court rejected this line of reasoning as entirely speculative. Fortunately, Catherine Albiston (UC Berkeley Law) and Laura Beth Nielsen (ABF / Northwestern Sociology)  have recently posted "The Procedural Attack on Civil Rights: The Empirical Reality of Buckhannon for the Private Attorney General," which is forthcoming in the UCLA Law Review.

Here is the abstract:

In 2001, in Buckhannon Board & Care Home, Inc. v. West Virginia Department of Health and Human Resources, the Supreme Court rejected the catalyst theory for recovery of attorneys' fees in civil rights enforcement actions. In doing so, the Court dismissed concerns that plaintiffs with meritorious but expensive claims would be discouraged from bringing suit, finding these concerns “entirely speculative and unsupported by any empirical evidence.” This article presents original data from a national survey of more than 200 public interest organizations that call into question the Court's empirical assumptions. These data indicate that organizations that take on paradigmatic public interest cases, such as class actions seeking injunctive relief against government actors, are the most likely to be negatively affected by Buckhannon. In addition, our respondents report that Buckhannon encourages “strategic capitulation,” makes settlement more difficult, and discourages attorneys from representing civil rights plaintiffs. We argue that these far reaching effects herald a shift away from private rights enforcement and toward more government power, both to resist rights claims and to control the meaning of civil rights.

Criminal Pro Se Defense

In a forthcoming article Erica Hashimoto (Georgia) explores an interesting corner of criminal law from an empirical perspective.  Although principally descriptive in its treatment of the data, Defending the Right to Self Representation: An Empirical Look at the Pro Se Felony Defendant finds:

"The data clearly refute both the assumption that most felony pro se defendants are ill-served by the decision to self-represent and the theory that most pro se defendants suffer from mental illness. Somewhat surprisingly, the evidence establishes that pro se felony defendants in state court do just as well as represented felony defendants, and the vast majority of pro se felony defendants - nearly 80% - displayed no signs of mental illness. The results of the study also provide an alternative explanation for the pro se phenomenon, suggesting that at least some defendants choose self-representation because of legitimate concerns about counsel. In short, the data in this Article expose the fallacy of the prevailing view of pro se felony defendants and demonstrate that the right to self-representation in fact serves a vital role in protecting the rights of criminal defendants."

USNWR Gaming and the Failure of Self-Regulation

I recently picked up the latest issue of National Jurist, which is a publication for current and prospective law students.  The cover story this month is "The Rankings Race: How Far Will Law Schools Go?"   

The article provides plenty of examples of how law schools attempt to game their U.S. News input variables, especially with regard to employment.  Michael Sauder, a sociologist at Iowa who has written extensively on rankings, offered instances in which law schools count alumni taxi drivers as "employed" graduates.  David Bernstein (GMU Law) cited a practice in which law schools have rounded up unemployed graduates and employed them for purpose of getting over the nine month USNWR hurdle.

The former President of the Associations of American Law Schools (AALS), William Hines, is quoted that "too many otherwise sensible law schools are engaged in questionable academic practices and may in some cases be borderline dishonest."  Two years ago, a similar damning story appeared in the New York Times.  See Alex Wellen, The $8.76 Million Maneuver.   

I don't think the legal academy appreciates the implications of all this well-documented gaming.  Not only are we cutting corners writ large in order to get ahead--and hence have zero credibility to emphasize the importance of honest, integrity, and ethical behavior to our students--but we are providing overwhelming evidence that we are incapable of effective self-regulation.  Right now, gaming pays.  Further, if you are a dean, and you refuse to aggressively manage your USNWR ranking, your job is in serious jeopardy.  But calling U.S. News a collective action problem is a cop-out.  Where is the ABA, our accrediting agency, to police this behavior?   

The table below, from the Measuring Outcomes paper by Andy Morriss and I, documents changes in graduate placement since U.S. News moved to the current employed at graduation/9 months methodology.  If these numbers are to be believed--and we would advise against it--these are bullish times for newly minted lawyers.  And the rising tide has raised all tiers.

Some of these remarkable gains may be the result of competition among schools to more effectively place, or at least track, their graduates.  If that is the primary explanation, then U.S. News has actually been a force for good, at least in terms of employment.  But a more likely explanation is that any employment--law firm associate, judicial clerk, public interest lawyer, taxi driver, waiter, insurance salesman, temporary law school RA, whatever--counts for the purposes of U.S. News. 

The biggest red flag is that while total employment at 9 months for all law school graduates surged from 83.9% to 91.6% between 1997 and 2006, first-time bar passage for this same cohort dropped from 83.0% to 78.6%, presumably due to the trend toward higher bar cut scores.  See pp. 13-14.

Note that the results from a February bar will rarely be available within 9 months of graduation. So, as we note in the paper,

it is likely that much of the increased post-graduation employment is in jobs that do not require a valid license to practice law and hence are unlikely to pay a premium for three years of legal education. At a minimum, this practice has the potential to seriously mislead prospective law students attempting to evaluate the relative costs and benefits of a legal education. It is also hard to believe that this type of reporting should be considered sufficient to satisfy ABA Accreditation Standard 509, which requires a law school to provide "basic consumer information" in a "fair and accurate manner reflective of actual practice."

As a self-regulating profession, it is time to get our own house in order.

Cross-Monitoring and Corporate Governance

I just finished reading a really terrific empirical study by Joanna Shepherd (Emory Law), Fred Tung (Emory Law), and AlbertYoon (Northwestern Law), entitled Cross-Monitoring and Corporate Governance.  In a nutshell, the authors show, after controlling for a wide variety of relevant control variables (most importantly corporate governance), that companies that rely on bank loans, as opposed to equity or public debt, post higher market returns.  Presumably, these results flow from the heightened monitoring and positive market signaling supplied by loan agreements.  Their multiple specifications do a nice job of ruling out alternative hypotheses.

This is quite an important paper for thinking about manager and director agency costs, especially in a world where portfolio diversification has reduced the incentives for shareholders to pay attention to annual elections for directors.  (It also is a excellent example of methodology for large panel data.)  Here is the article abstract:

We take the view that corporate governance must involve more than corporate law. Despite corporate scholars' nearly exclusive focus on corporate law mechanisms for controlling managerial agency costs, shareholders are not the only constituency concerned with such costs. Given the thick web of firms' contractual commitments, it should not be a surprise that other financial claimants may also attempt to control agency costs in their contracts with the firm. We hypothesize that this cross-monitoring by other claimants has value for shareholders. 

We examine bank loans for empirical evidence of the value of cross-monitoring. Our approach builds on prior empirical work on the value of good corporate governance, to which we add data on the presence of bank loans and their interactions with free cash flow, governance indices, and individual corporate governance provisions. We find strong evidence that bank monitoring adds value. In effect, bank monitoring can counteract somewhat the value-decreasing effects of managerial entrenchment. Bank monitoring may substitute for good corporate governance.

Abstract - Traffic Tickets & Local Government Budgets

Thomas Garrett (Federal Reserve Bank of St. Louis) and Gary Wagner (University of Arkansas) ask, "Are Traffic Tickets Countercyclical?" Here is the abstract:

There is anecdotal evidence that local governments use traffic tickets to generate revenue. Using panel data for North Carolina counties, we examine whether changes in local government revenue influence the number of traffic tickets issued. We find strong evidence of an asymmetric response by local governments. Specifically, positive changes in revenue have no effect on traffic tickets, but negative revenue changes increase the number of traffic tickets issued. A one percentage point decrease in revenue yields a 0.38 percentage point increase in traffic tickets. We calculate that traffic ticket revenue supplements a low percentage of local revenue losses.

The full paper is available here on SSRN. (H/T theNewspaper.com).

The Gender Wage Gap: Spouses with "Intense Jobs"

Here at Indiana Law, I teach a course called The Law Firm as a Business Organization. In a nutshell, the course provides students with an overview of the historical, economic, and sociological factors that have shaped—and continue to shape—the modern legal marketplace.   

This year, thanks to the dedication and hardwork of my colleagues, Ken Dau-Schmidt and Jeff Stake, my course includes extensive data on Indiana Law graduates; in other words, current IU students get to study the career trajectories of IU alumni.  This has been an interesting experience as both a teacher and a researcher.  The first study, which was explicitly designed to facilitate comparisons with the famous Michigan alumni dataset, is now available on SSRN: "The Pride of Indiana": An Empirical Study of the Law School Experience and Careers of Indiana University Law Alumni. (Ken and Jeff will also be presenting these results at the CELS conference in Oct.)

As we moved through some of the data last week, I noted some interesting breakdowns associated with the variable "% Spouse has intense job."   This item reflects the percentage of respondents who are married to professionals with demanding jobs (e.g., other lawyers, doctors, business executives).  Here is a summary of the "intense job" data along with some relevant additional variables:

As shown in the table above [click to enlarge], male and female Indiana Law graduates in both the five- and fifteen-year cohorts are fairly comparable on some dimensions (shown in blue): % married, size of families, % of spouses who are homemakers, and total household income.  But there are consistent statistically significant differences along three dimensions (shown in green):