Give the People What They Want...

Do you want to increase your SSRN downloads? If so, you should consider publishing a law school ranking. (If not, congratulations, you are far healthier than most of your colleagues. Of course, it does prompt one to wonder why you’re reading this post.)

In April 2005, I participated in the Indiana Law Journal Symposium on “The Next Generation of Law School Rankings” organized by Paul Caron and Rafael Gely of Cincinnati and ELS blog’s own Bill Henderson and Jeff Stake of Indiana. Invited to discuss ranking methodologies, I opted to create my own ranking. For me, the choice of focus was obvious: Empirical Legal Scholarship (ELS). Why? An ELS Ranking identifies schools leading in this burgeoning field and perhaps incentivizes others to invest in it.

The paper ranked each of the top 41 schools in US News based on an ELS-score that I constructed. The score included three variables: professors with social science doctorates, professors holding secondary appointments in social science departments, and articles in ELS-oriented publications. First posted on SSRN on August 9, 2005, the paper has been viewed more than 4,000 times and downloaded more than 860 times. (By contrast, my paper with Jeff Berger on judicial entrepreneurs posted at the same time has been downloaded only a fraction of that number – help here!) UC-Berkeley and George Mason each boasted on its webpage that it tied for first. Legal Affairs and various law blogs all provided short discussions of the piece. Obviously, law professors and their groupies love law school rankings.

I have revised, updated, and expanded the ranking and will post the new results over each of the next four days. The ranking now includes the top-50 schools in US News and looks at faculty as of 2006-2007 (as opposed to 2003-2004 in the original). This week, I will describe each measure and the new results, focusing on educational background and training on Tuesday, secondary appointments on Wednesday, and empirical research output (the only measure that has been completely changed) on Thursday. And, the 2006 ELS Ranking will be available on Friday.

My purpose here is not simply to navel gaze but also to seek input on the measures. Operationalization is hard. And the power of any empirical results necessarily turns on the soundness of those measures. I invite comments on the value of rankings (particularly given the substantial research costs including time away from other work) as well as thoughts on the components of the ELS ranking. Thanks to Jason Czarnezki for inviting me to be a guest blogger and to Geoff Turvey for exceptional research assistance.

Comments

[Jason Czarnezki]

Due to the costs of collecting even more data, most rankings other than U.S. News focus on the Top 40-50 Law Schools. However, I think we may lose inmportant information as a result. If you want to incentivize ELS, you need to give lower ranked schools credit if they invest in it. Also, it is hard to quantify the impact of ELS as a movement and know whether schools took AALS (who had their annual meeting on the topic) up on their explicit suggestion that ELS is worthwhile, without knowing if scholars at the other 150+ law schools do ELS-type work. Could a lower ranked school gain a national reputation by focusing on ELS? Like Vermont, Pace, and Lewis & Clark have gained a national reputation in environmental law. Having empirical scholars may be too costly for this to occur (hiring stats knowledge faculty or JD/PhDs, costs of continual traning like ICPSR or ELS Wksp, stats software costs, etc.), but I wish I had a better sense outside the Top 50 of who was doing high-quality empirical work. Among the Top 50, the major players seem fairly obvious due to conference affliations, etc. Also, if the costs of ELS are high, it would seem to me that there might be a correlation between ELS rankings and U&S News Rankings or endowment. Is this true?

[frank cross]

I don't think you should punish people (i.e., me) for publishing empirical work in law reviews. Maybe you could count top twenty publications in the score?

[Gordon Smith]

Jason: "Could a lower ranked school gain a national reputation by focusing on ELS? Like Vermont, Pace, and Lewis & Clark have gained a national reputation in environmental law."

No. Two reasons: (1) ELS is hot, and young scholars doing good work would be snapped up by bigger schools before their home institution has a chance to build a critical mass and a reputation for ELS; and (2) ELS is too diverse to cohere as a specialty.

[Corey]

Yeah, that article was part of a whole 400 page Indiana Law Journal issue on law rankings last year. Significantly increasing our exposure and almost assuredly our "most cited journal" ranking. Was that a cynical exploitation of the self-reflecivity of legal scholarship, or noble gesture towards meeting market demand for talk of rankings? You decide.

Articles Editor, Indiana Law Journal

[Tracey George]

Frank Cross writes: "I don't think you should punish people (i.e., me) for publishing empirical work in law reviews."

Frank expresses a view held by many others that the original empirical research measure in the ELS ranking was flawed. I've changed this measure dramatically (as will be explained in more detail on Thursday), but shortcomings remain. Please return on Thursday for more discussion on this point.

[Anono]

If not, congratulations, you are far healthier than most of your colleagues.

An accurate assessment, but prepare to face the wrath of Brian Leiter!