Where is the Moneyball at Moneylaw?

Over at Moneylaw, a new blog on "the art of winning an unfair academic game" (a reference to the subtitle of Moneyball), Jim Chen asserts that the legal academy overvalues pedigree and undervalues performance, thus thwarting the emergence of a true academic meritocracy.  Jim then writes:

For my part, I offer my own tentative list of core principles for evaluating academic talent, whether on an individual or an institutional basis:

• Pedigree never matters.  Performance always does.
• In gauging performance, you can measure anything you want except other professors' opinions.
• In other words, no academic ratings system is valid if it depends in whole or in part on a subjective survey of academic reputation.

What, exactly, is Jim's objective here?  To structure a more theoretically coherent system of merit that will crush all existing ranking systems by sheer force of reason?  And surely, performance cannot be "anything you want" if the point is to "win an unfair academic game"--you need some criteria to know your making progress (or "winning"). 

In baseball, there are balls and strikes, hits and outs, and ultimately wins and losses.  But in the legal academy, it is much harder to operationalize useful constructs of performance.  Assuming we throw out reputation, as Jim suggests, and focus on productivity (certainly under the rubric of "performance"), lots of measures seem relevant:  articles produced, citations by courts, citations by scholars, SSRN downloads, journal placement (using, say, the W&L impact measures).  And if we can agree that some or all of these are relevant in this "game" we are engaged in, then how shall they be weighted?  Surely, the folly of this enterprise is self-evident.

It is rare that a junior scholar has the opportunity to utter these words to a scholar of Jim Chen's stature, but here it goes:  Jim, you need a theory. 

For a coherent theory on how Moneyball principles can be fruitfully applied to the law school world, and for some interesting data, you are going to have to read past this jump.

Before diving into the theory, let me offer a data point that supports Jim's contention that the legal academy suffers from an elitism that corrupts our perceptions of talent and ability.  (At bottom, I am quite sympathetic to Jim's position.)

A friend of mine is a visiting professor at a non-prestigious regional law school. He/she has a dozen journal publications with a good placement record (including peer-reviewed articles) and two (yes, two!) books on legal subject matter published by major academic presses and another two under contract (we are talking top Ivy League presses).  His/her teaching evaluations are stellar.  Yet, he/she has had hard time getting AALS interviews and a tenure track job because he/she attended a non-prestigious regional law school, was not on law review, and (no surprise) did not clerk for a judge.   Yet, his/her entry level publication record blows away most tenure files at Top 5 schools.   (By the way, with his/her permission, I will pass along his/her c.v. to any interested hiring chairs, but not to the merely curious.)

But this post is about Moneyball and Moneylaw.  So the question is whether this elitism--akin to the scouts who recruited Billy Beane based on his build and athleticism and overlooked his inability to hit a curve ball--results in predictable over and undervaluation of talent, thus creating opportunities for arbitrage in the market for law professors.  In their masterful review of Moneyball, 82 Tex. L. Rev. 1483 (2004), Paul Caron & Rafael Gely made a compelling case that appointments committees were indeed employing heuristics, such as school attended and fancy clerkships, that had no bearing on scholarly productivity.  But they did not lay out a systematic vision for how these lessons could be deployed to build a successful law school or law faculty.

Some Realism on the Marketplace

Because success in the law school universe is not measured by a wins and losses, the first analytical task is to frame out a coherent, tangible vision for how one measures success.  Here, I am fairly comfortable with money:  students are willing to pay relatively high tuition for the education and opportunities at law school X, and alumnus are sufficiently grateful for the transformative experience they received--and here I am not being remotely hyperbolic--that they are willing to underwrite X's mission and subsidize this opportunity for future generations.   

Obviously, this strategy is not new.  It produced the endowments at Harvard, Yale, Virginia, Michigan, Chicago, Columbia, and NYU, etc.  (see Leiter's recent breakdown.)  These institutions are now free to pursue the Alex Rodriguez's ($25M/yr) or the Manny Ramirez's ($20M/yr) in the entry and lateral markets.   Further, the rules for keeping your reputation are much different--and more lenient--than the rules for building one.

But here, it is important to recognize that widely held perceptions of law school quality are more than fictions dreamed up by commercially oriented news outlets or the minds of vain law professors (perhaps vain is redundant since we are all vain).  Rather, these perceptions have, for decades, facilitated highly desirable employment options for students that will often last past the first job.  Twenty years after graduation, a Harvard JD is still impressive and still opens doors.  Ceteris paribus, the degree itself confers hedonic and (likely) financial benefits to the recipient. 

Further, as a historical matter, the hierarchical nature of legal education, and its concommitant effects in various labor markets (students, judicial clerks, lawyers, law professors, etc) did not emerge in 1987 when U.S. News & World Report published its first law school rankings.  For example, below is a table that lists (a) the Top 9 U.S. law schools according to a survey conducted by Change Magazine (Winter 1974-75), which asked 134 law school deans to name the Top 5 law schools (the schools below were named by at least 10% of the deans) , and (b) the current U.S. News ranking:

How could it be that the very same schools can be at the top of the heap, albeit in a slightly modified order, after 32 years--an entire career for most professors?  The answer is simple:  labor markets.  Sure, law schools nominally compete.  But schools with the largest endowment can behave like the NY Yankees and buy the best talent to perpetuate their elite status.   For example, in a 1957 article in the Journal of Legal Education that discussed the distinction between a regional and a national law school, Professor Harry Jones of Columbia provided the following apt baseball analogy:

Twenty years ago, it was common usage to discuss "local" and "national" in terms of analogy to "minor leagues" and "major leagues."  The analogy never held in individual cases, since "local" schools had their Mickey Mantles and every "national" school its full share of .200 hitters; but there was much to it: the average salary of a "national" law school professor was about twice the average salary at a typical "local" school, and the "nationals" regularly replenished their faculties by a raiding process comparable to the major league draft.  The most striking legal development since World War II has been a narrowing of the "national"-"local salary gap [due to taxpayer funding of public law schools].

Jones, Local Law Schools vs. National Law Schools: A Comparison of Concepts, Functions, and Opportunities, 10 J. Leg. Educ. 281, 289-90 (1957).

The dynamic of national/regional law schools and a thick market for lateral talent is certainly no less true today.  But we can tease out a couple of specific lessons from the enduring nature of this equilibrium.

1. Money matters.  Schools with significant resources can buy the most coveted talent.  Moreover, there is clear empirical evidence that lateral hires are, as a group,  more productive (in terms of quantity and quality of placement) than faculty who were hired as entry levels--i.e., people hired, no doubt, partially on the basis of high law school grades, law review, and fancy clerkships.  See Lindgren & Seltzer, The Most Prolific Law Professors and Faculties, 71 Chi.-Kent L. Rev. 781, 805 (1996) ("The most striking finding of this study is that nineteen of the top twenty-five individual publishers are lateral appointments.").
2. There is little or no market discipline at the top.  If you are at the top of the heap, there is no market penalty if your scholars treat teaching and students as annoying distractions.  Donors will still donate because they want of a piece of your hallowed reputation. Further, the reputation provides a signal that coordinates the market for legal employers and well-credentialed students.  Gaps in education are tolerated because the JD pedigree itself confers financial and hedonic value to students and employers.  See Korobkin, In Praise of Rankings: Solutions to Coordination and Collective Action Problems, 77 Tex. L. Rev. 403 (1998) (positing that rankings permit labor markets to coordinate and clear and are not estimations of educational quality); Korobkin, Harnessing the Positive Power of Rankings, 81 Ind. L. J. 35 (2006) (same).

There is another analysis that applies to law firms who want to play Moneyball/Moneylaw and get better outcomes than rankings-driven recruitment strategies.  But that is the topic for a separate post.  (If any law firm managing partners are interested, I authored a detailed memo-to-file that explains how quantitative methods can identify systematic biases, and thus inefficiencies, in the recruitment of associates; I am very serious! email me.)

So here is the nut of the problem: A school cannot claw its way to the top of the heap by a scholarship-first strategy because it lacks the bankroll; to do so is to subsidize the creation of talent that will migrate to higher ranked schools in pursuit of more money or prestige.  In reality, the money just won't show up in time (or at all)  to keep your home grown talent.  Such a strategy, which is arguably dominant in legal academia, is misguided.  This herd mentality is akin to the scouts who unanimously predicted Billy Beane's success as a player.

An Institutionally Coherent Strategy

With this understanding of the market in place, if a law school hopes to someday enjoy the blessing/curse of too much money, it has to add value to the personal and professional lives of its current and former students.  And that requires careful attention to the nuts and bolts of a law school over a prolonged period of time. (Perhaps it is no coincidence that I recently blogged about the king of buy-and-hold-value, Warren Buffett.)

Here are a few things that might help launch some sucessful careers, impress alumni, and lay the foundation for higher donations in the future:

• Scholarship as a Vehicle for Teaching. Hire academically minded faculty who evidence creativity in the classroom.  I suspect it is possible to tone down some of the usual fancy school / clerkship / graduate degree fixations and find undervalued candidates who see connections between their scholarship and their teaching.
• Mentor Students.  Hire professors who are willing to mentor and advise students--i.e. the people in the building who will one day be elected officials, business leaders, judges, and ethical and successful law partners.   At t+10 or t+20, getting an endowed chair for such professors will be much easier.
• Create a Magnetic Culture.  Hire professors who buy into a vision of helping create the next generation of lawyers--and reduce their teaching loads so the mentoring and scholarship and alumni relations are manageable rather than burdensome.  Faculty engagement is more important than a wider array of course offerings.  Who knows, some of your superstars may actually turn down lucrative lateral offers to participate in this endeavor.
• Invest in Curriculum.  The legal profession is changing rapidly.  Reward faculty for thinking through a curriculum that prepares students to hit the ground running.  An example is UCLA's Business Law certificate program, which sequences business courses in an intelligent way and draws upon local practitioners (I suspect many alumni) to teach capstone skills courses.  It is my impression that LA employers dig a little deeper to hire students who have completed this program.  Here, assembling accurate empirical evidence is essential.   
• Invest in Career Services.  The Law School Survey of Student Engagement  consistently documents career services as the Achilles' Heel of every non-elite law school.   Here the task is two fold: (1) systematically identifying employers who want to hire your well-prepared graduates, and (2) equipping students with the professional saavy to close the deal. 

If the point is to augment future revenue streams to pave the way for more and better talent, are more publications  in fancy journals next year going to achieve that goal?  And don't you want some cultural component that mitigates lateral raiding--i.e., gives faculty a non-financial reason to stay and finish the job? 

It is worth noting that the most profitable law firms did not build their franchises through heavy reliance on lateral markets; rather, firm culture reinforced an ethos of superb work product and client service.  Further, when the Boston Red Sox offered to make Billy Beane the richest GM in baseball history, he turned it down because last time he made a decision based purely on money--rejecting a baseball scholarship to Stanford to sign a $125, ooo minor league contract--it was the wrong decision and he regretted it.

An Misguided Strategy Coherently Executed

There is a perception among many Moneyball/Moneylaw devotees that law schools are failing to avail themselves of clear opportunities to hire undervalued talent.  I blogged about this topic last year at Conglomerate. See "A Numbers-Driven Appointments Committee."   Frankly, I have a suspicion that Dan Polsby, et al. are running some regressions at George Mason.  Of course, their prodigious success (in SSRN downloads; they still have to room to climb in US News) may be a function of a simple left/right arbitrage.  Of course, many law schools have an appetite for conservatives if they are sufficiently well published, but GMU probably has a culture that has helped it retain some of its talent.   In general, however, most law schools seem to emphasize the same heuristics of school/grades/clerkship/publications for the entry level market. 

If a law school wants to systematically identify undervalued future scholars, I am available to create the requisite dataset and run the necessary regressions--but I am expensive.  What I cannot guaranteed, however, is whether a law school can parlay that talent into any meaningful measure of long term institutional success.  A scholarship-first strategy is misguided, whether done through Moneyball/Moneylaw or the usual elitist folk wisdom.  Not only does it shortchange students (thus disturbing our restful sleep), there are solid theoretical and empirical reasons to suggest that it just doesn't work.

Is it any wonder that both faculty and candidates hate the AALS market? 

Comments

[generic viagra online]

i like this part of the post:"The dynamic of national/regional law schools and a thick market for lateral talent is certainly no less true today. But we can tease out a couple of specific lessons from the enduring nature of this equilibrium." is verygood

[custom research paper]

Oh! this site has really nice & awesome information. Thanks.

[QILisa]

The essay writing just about this topic, all would notice at the custom writing service. Buy the research papers or custom writing just about this post.

[anona]

Thanks for your compelling post re money-law - wow, that was a lot to digest in one posting. Money-law is a favorite blog of mine, so I was at first a bit taken back a bit by your post taking Jim and the Money-Law crew to task. However, it appears (from your prior post) that you are generally on board with Jim's idea but are appropriately challenging him to refine it.

A few quick takes on your post:

1) I think that this a "ships passing in the night" problem to a degree - Jim seems to me to be acting as more of an advocate and you are thinking about it from a social science and theoretical perspective and
a perhaps more strictly moneyball practical standpoint - does it get "bang for the buck" in scholarly productivity for law schools. I think that Jim likely mixes in a concern
over societal fairness and disdain for elitism along with the practical
money-ball "does it work" perspective. You appropriately challenge him to provide the causal mechanism.

2) The causal mechanism - you make a good point that this is actually a pretty complicated situation. Assuming that a law school desires scholarly productivity (which I address below as a problem),obtaining good faculty (often with very little helpful
information) is difficult, and retaining really productive faculty may be even
harder, especially when a law school has limited resources relative to other schools. However, there are a variety of ways that schools can do this (or make retention more likely)- there is enough on this point for an extended discussion - perhaps more appropriate for another post or exchange. I'd be happy to address these "moneylaw retention strategies" if anyone is interested. But your concern over whether moneyball works in law school rankings is noted. Law schools dont play for the annual world series - it's a long term game with a great deal of sluggishness (as you demonstrate).

3) However, and this is a big 'however', the alternative to moneylaw - for all practical purposes, seems to be the type of "elitist folk wisdom" you denounce in your previous post on Conglomerate. I think that Jim actually makes a pretty modest claim that "past performance predicts future performance" better than the existing default axiom that "elite pedigree predicts performance". Jeff Harrison's post on moneylaw and his blog suggests some research of his to support this (although I cant
claim much here, as I haven't read it). Additionally, my experiences in academia seem to corroborate the "past performance as a predictor of future performance" idea that moneylaw promotes.

4) As you might guess, on your previous post on Conglomerate, I am the anon_cd commenter. I stand by my anon argument as to why schools dont play moneylaw - they dont *really* want to win and there isn't consensus as to what win is anyway. I dont mean everybody at a law school - but enough faculty - simply want to pay lip service to the importance of scholarship and productivity. They have strong personal incentives for perpetuating the status
quo (pedigree rules). Look at law schools and how they work - it's quite apparent - they hire pedigreed people like themselves (see Leiter's analysis). It retains the currency of the pedigree degree. What's the point of having a degree from Yale/Harvard/Stanford if it's all about scholarly performance after graduation? Why would a pedigree grad in a power position to hire or promote/tenure choose to change to scholarly performance merit system that essentially obviates or works to undercut a clear market advantage that he enjoys? Fairness? I think that we know better than that.

5) As person with a similar record (to your friend) who is also experiencing little success on the AALS scene, I empathize with your friend's situation. I think that situations like this are perhaps what drives money-law thinking and Jim's convictions. As you might guess, I'd also be happy to offer my record to interested hiring committees - just provide your email here and I'll send it to you directly. I challenge them to consider me under objective moneylaw criteria against their other applicants.

To be quite frank I think that while we are on the cusp of understanding the problem of elitism in law school hiring, we have a long way to go before people are really willing to draw a line in the sand and stand up
for the proposition that performance matters. Perhaps your planned (although yet unfunded study) will go some ways toward addressing this issue.

[Anon.LawProf]

You post, Bill, is infused with a palpable sense of both humility and humanity. Thank you very much for it.

[Vic Fleischer]

Great post, Bill. But I wonder (focusing on the end of your post) if you underestimate the institutional value of having great scholars who migrate up. They bring energy to the school and can help create a culture of productivity that benefits everyone. And I suspect they actually tend to be better teachers, too, than scholars who don't migrate.

If that's right, the moneyball strategy might be to find scholar/teachers with great potential who will enhance the school's reputation ... but also who might tend to stick in one place (people w geographic ties, couples, women - see Rosa Brooks' article in the recent YLJ pocket part). Some will leave anyway, but even if that happens, the school still gains something in the meantime.

Just a moneyball theory.