Applying Empirical Methods to Legal Scholarship

I'll start off this week's guest blogging by talking about a joint project (with Funmi Arewa (currently Case, moving to Northwestern)), Bill Henderson (Indiana), and Ken Dau-Schmidt (Indiana)). This project is in an early stage, so I am hoping to take advantage of the expertise of the ELS crew and readers in the comments to help us resolve some problems in structuring our data.

We're interested in the evolution of legal scholarship - so this is really "empirical scholarship about legal scholarship." We have two major goals in studying it. First, legal scholarship has changed dramatically over the period for which we have data. To cite just one example, "doctrinal" work is largely seen as out of favor today, compared to the 1950s and 1960s. Another example of change is the proliferation of journals, including the appearance of faculty-edited journals in specialty areas like empirical work. The increasing importance of the U.S. News rankings (together with quite a bit of anecdotal evidence that student editors at some journals are using USN rankings of authors' institutions as a screenign device). We''d like to get a handle on what has happened. Second, there is a ton of folk wisdom out there about law review placement and we'd like to know what's true and what's not (e.g. is there a "home field" advantage in placing articles at an author's home institution?). (If comments indicate interest, I can discuss the reasons we think legal scholarship is worth studying, otherwise I am going to take it as a given that trends in legal scholarship is a worthy topic for study.)

To study the evolution of scholarship, we've obtained a dataset of every publication included in the Index to Legal Periodicals from 1928 to 2005. We plan to match this data to data from other sources (e.g. the AALS Directory of Law Teachers as a source for author credentials). Right now, however, we're figuring out how to get the ILP data into shape. The biggest problem is dealing with the ILP's approximately 1700 subject codes. These are a blessing (someone else has categorized all the data) but also a problem, since there are a lot of them and they don't necessarily fall into categories as neatly as they would in an ideal world. Each article can have multiple subject codes (generally, most have 4 or fewer). Consider the law and economics classifications, which ILP calls "economic jurisprudence."

Here's what the ILP includes:

Economic jurisprudence

          See also Coase theorem; Public choice theory

Economic jurisprudence/Administrative law

Economic jurisprudence/Antitrust law

Economic jurisprudence/Bankruptcy

Economic jurisprudence/Behavioral economics

Economic jurisprudence/Civil procedure

Economic jurisprudence/Commercial law

Economic jurisprudence/Compensatory damages

Economic jurisprudence/Conflict of laws

Economic jurisprudence/Constitutional law

Economic jurisprudence/Contracts

Economic jurisprudence/Copyright

Economic jurisprudence/Corporations

Economic jurisprudence/Criminal law

Economic jurisprudence/Criminal procedure

Economic jurisprudence/Environmental law

Economic jurisprudence/Health care industry

Economic jurisprudence/Inheritance and succession

Economic jurisprudence/International law

Economic jurisprudence/Labor law

Economic jurisprudence/Patents

Economic jurisprudence/Personal injuries

Economic jurisprudence/Products liability

Economic jurisprudence/Property

Economic jurisprudence/Punitive damages

Economic jurisprudence/Securities

Economic jurisprudence/Settlements

Economic jurisprudence/Telecommunications

Economic jurisprudence/Torts

Economic jurisprudence/Trade secrets

Economic jurisprudence/Trademarks

For some purposes we'll keep the 1700+ classifications. Comparing "Economic jurisprudence/torts" articles to "Torts" articles that don't include an economic jurisprudence code would be one way to compare law and economics work to non-law and economics work, for example.

But we think we need a more manageable set of subject codes for other analyses. One possibility is to simply reduce the 30 or so "economic jurisprudence" classifications to a single "economic jurisprudence" subject variable. Another is to fold in the "see also" codes of "public choice" and "Coase theorem". A third is to undertake some form of cluster analysis to see what codes tend to go together and try to construct meaningful codes from clusters of similarly-coded articles. We'd love to hear about experiences people have had reducing large numbers of classifications, the techniques they've used to avoid imposing preconceived structures on the data, and the like.

Comments

[Matt Quick]

The Directory of Law Teachers is available on Westlaw, but not in a form that makes it easy to do empirical analysis. Is the Directory available somewhere else as a more convenient database?