Empirical Legal Studies

Building a Relational Database

If I had to write an essay on my summer vacation, this year's topic would be "Building a Relational Database."  As I noted a few months ago, the LSAC funded our proposal for a large-scale longitudinal analysis on the production and content of legal scholarship (my co-PI's include Funmi Arewa, Ken Dau-Schmidt, Peter Hook, and Andy Morriss).  The most difficult task in this endeavor is cost-effectively digitizing biographic information on every law professor who ever appeared in an AALS directory (from 1922 to 2007) and linking these individuals to the detailed bibliographic records contained in the Index of Legal Periodicals from 1928 to the present.

How does someone tackle a job this big?  My legal training has negligible value here.  It it really all about planning and collaboration among a team with a wide array of skill sets.  With the input of a talented database expert, Mark Newton, and our co-PI, Peter Hook, who is a law librarian with graduate training in information sciences, we are developing and implementing a process that--we theorize--will (a) maximize the capture of information and complex relationships, and (b) minimize key strokes and errors. 

So far, one of our greatest assets is the excellent workforce here in Bloomington.  We currently have a research team that includes four graduate students from the School of Library and Information Sciences (SLIS), two excellent college graduates who are headed to law school in the fall, and a brilliant University of Chicago undergraduate with an eagle eye for detail.   These talented folks are closest to the data and provide enormous insights that affect our planning process. 

Although the relational database itself will not produce a scholarly paper, I am really grateful for the opportunity learn a lot of new technical and managerial skills.  I am also developing a deeper respect for expert knowledge and team work.      

Markets and Discrimination

Jake Gersen (Chicago) has written Markets and Discrimination, forthcoming in the NYU Law Review.  I read this paper in draft form and have seen it presented as well... I highly recommend it.  The Abstract:

Despite decades of scholarship in law and economics, disagreement persists over the extent of employment discrimination in the United States, the correct explanation for such discrimination, and the normative implications of the evidence for law and policy. In part, this is because employment discrimination is an enormously complex phenomenon, and both its history and continued existence are closely linked to politics and ideology. However, some portion of this dispute can also be traced to the incomplete use of empirical evidence. Most economic theories of employment discrimination imply empirical relationships between discrimination and the market structure of particular industries and characteristics of their workforces.

Yet empirical work has most typically focused on either specific industries or the economy as a whole, and little systematic evidence about market structure and patterns of actual employment discrimination claims exists. This Article compiles and analyzes an original data set comprised of industry-specific measures of employment discrimination claims, market conditions, and labor force characteristics. In so doing, this Article contributes to an emerging literature that tests the core theoretical positions in the law and economics of discrimination literature, which in turn promises to advance understanding of both the causes of and remedies for employment discrimination.

Home Court Advantage?

Lee Epstein has an interesting new post at Balkinization, see here, discussing whether Supreme Court Justices more often affirm cases that come from the circuit to which they previously served (i.e., home court) before their elevation to the Supreme Court.   Based on her post, it sounds like this could be another interesting paper from Professor Epstein.

The Role of Judicial Ideology in Economic (Tax) Cases

Noting that "study after study confirms a strong correlation between judges' political preferences and their behavior in civil rights/liberties-type cases, but researchers have only rarely identified an association between politics and decisions in economics cases," a recently circulated paper by Nancy Staudt (Northwestern), Lee Epstein (Northwestern), and Peter Wiedenbeck (Wash U) asks the interesting question: "Why do judges appear to stand above politics in the areas of the law that are rife with conflict and controversy in the other two branches of government?"  From the abstract:

"Lawmaking in the context of taxation, bankruptcy, securities, antitrust, corporate law, to name just a few examples, is highly political in both the legislative and executive branches, as many empirical scholars have documented. For this reason, we seriously question the claim that judges are unique in that they have no political or ideological preferences when it comes to business and finance. Our conjecture is that the null findings in the literature are due to the technical difficulties associated with uncovering politics in large-N studies addressing economics decision-making rather than to a lack of judicial interest in these issues. But this is precisely the question we investigate here."

Although the authors characterize their findings, drawn from U.S. Supreme Court tax cases decided between 1940-2005, as "preliminary," they conclude that their findings support rejecting "the null hypothesis that politics plays no role in judicial decision making that involves business and finance."

Does the Research Question Drive the Methodology or Does Methodology Drive the Research Question?

As I was pondering possible future writing projects over the past few days, I began thinking about the different ways to approach interdisciplinary scholarship.  A greater number of scholars in recent years seem to let methodology drive the research question.  For instance, a researcher might use a particular type of game theory to explain behavior in a broad swath of substantive legal areas.  That scholar might become an expert at game theory, but may have no special expertise in any particular subject matter such as torts or criminal law.  An advantage of such an approach is that the scholar becomes very familiar with a particular set of methodological tools that may permit a fresh look in a variety of substantive legal areas.  On the other hand, a disadvantage of such an approach may be that, without special expertise in the subject matter of the project, a scholar may have more difficulty putting his or her findings in context or may not be familiar with the exceptional attributes of the subject matter being studied.

My preferred approach to scholarship is to allow the research question to drive the methodology.  Rather than repeatedly employing a particular methodology in a wide variety of subject areas, I look for questions that I personally find fascinating and to which I do not know the answer.  Although most of my work is empirical, such an approach requires me to venture outside of my comfort zone for some of my writing projects, including the legal history project that I discussed over the weekend with respect to the life and jurisprudence of Pierce Butler.  One advantage to this approach is that, unlike methodologically-driven research, I find that it is easier to gain a broad perspective and expertise about a particular legal subject matter or legal institution.  For me, all of my research questions center around the federal judiciary and the Supreme Court, and I by no means consider myself a methodologist.  An obvious disadvantage is that scholarship driven by a research question often requires scholars to use methodological tools that they have not employed previously.  It therefore often takes greater time, patience, and care to allow the research question to drive the methodology.  (The exception, of course, would be strictly doctrinal scholarship, in which the research question still drives the project but there is no need to learn or employ new methodological tools because law school, practice, and/or a clerkship provide ample training for such scholarship.) 

I do not think that one approach is better than the other, and it may be that most scholars employ a hybrid of the two approaches when approaching their scholarly pursuits.  I wonder which of the two approaches is more risky though, especially for untenured professors?  Which approach is more common in the legal scholarly community?  Are there other approaches that I am simply omitting from the discussion?  Although this post is admittedly abstract, I think it is worth addressing some of these questions as these are the types of issues that new law professors who want to do interdisciplinary work face as they enter the legal academy. 

The Private Sector and the Power of Social Network Analysis

After my one week of immersion in Social Networking Analysis, I became convinced of two propositions.

1. This powerful set of tools can reveal key dynamics within any organization, industry, or social setting, broadly construed, and thus formulate more effective polices/strategies/tactics; this power can be harnessed for good or evil. 
2. Because of the nature of the required data, university researchers are at a huge disadvantage.  The most sophisticated research in this area is being done by the private sector, which has fewer privacy strictures (i.e., human subjects/IRB regulations) and a strong motive to fund proprietary research.

In contrast to other social science methods, social network analysis requires complete data of a social setting rather than a sample.  Thus, if your data is derived from survey data, you need response rates of 90 to 100%, which are rare.  Yet, in this digital age, monitoring computer activity is a very cheap and effective to assemble the required dataset. 

For example, emails provide an excellent record of who communicates with whom; and content analysis can be performed by software.   This is great data, but for most research questions, the human subjects issues will be intractable.   Because this research has important commercial implications, the private sector is well positioned to pick up the slack.   

Prediction:  Expect to see more researchers leaving the academy to pursue research opportunities in the private sector.   It is not just about the money--their new employers are likely to have access to better data.

The Future of Personal Papers and Archival Research?

I spent an enjoyable day in the personal papers of Justice Pierce Butler, which are housed at the Minnesota Historical Society.   The Butler family kindly gave me permission to use these restricted papers for an article on Justice Butler that I am writing for a conference at Vanderbilt Law School next spring.

As I was reading the hand-written correspondence between Justice Butler and his various family members, friends, and colleagues, it struck me that archival research may look a bit different in the future.  Justice Butler's correspondence tells such a rich story, both about Butler's professional accomplishments and his family, but also about his temperament and passions.  Today, in contrast, so much correspondence takes place electronically, primarily via e-mail, and it is rare to receive a hand-written letter with the exception of greeting cards at holidays.  Unlike the letters of the past, electronic communications as a general matter tend to be shorter, less personal, and do not reveal quite as much about the relationship between the sender and the person to whom the correspondence is addressed. 

Besides enjoying the content of the documents, I kept asking myself the question of what the personal papers of the future will look like?  Will prominent individuals donate their hard drives to document repositories in the future?   Will we, as researchers, be searching servers full of electronic documents rather than reading the rich hand-written notes of the past?  If the answers, as I suspect, to the foregoing questions are yes then I believe that archival research is bound to be less interesting (but perhaps easier) in the future.   

Dababase on Wrongful Convictions

In a column in Law Technology News that reviews new law-related web resources, lawyer & blogger Robert Ambrogi reports on a new database at University of Texas that tracks wrongful convictions.   If I were a criminal law scholar trying to find a good empirical project--or, more aptly, looking for a project that could improve the functioning of our criminal justice system--I would be giving this resource a very close look.  Here is Ambrogi's description:

The Tarlton Law Library at the University of Texas School of law has created the Actual Innocence Awareness Database to track developments related to wrongful convictions. The database contains citations (and links, when the materials are online) to popular media, journal articles, books, reports, legislation and Web sites. Materials are classified by the primary causes of wrongful conviction: forensics/DNA, eyewitness identification, false confessions, jailhouse informants, police or prosecutorial misconduct and ineffective representation. There is also a general category, and the entire database can be searched.

Kudos to the Tarlton Law Library for assembling this database.

The Geographic Network of Large Law Firms

As I noted earlier this week, I attended a 5-day ICPSR course on Social Network Analysis.  Yesterday, I decided to take some of the network  software out for a spin by analyzing a dataset I have on branch offices of National Law Journal 250 firms (the 250 largest firms by number of lawyers) for 2006. 

The chart below [click to enlarge] is a "2-mode" network sociogram that depicts degrees of interconnectedness between cities (blue squares) and NLJ 250 firms (red circles). 

Here are some quick interpretative guides:

• Cities.  You'll note that lots of firms tend to have offices in New York, Washington, Chicago, L.A., and San Francisco. Hence, the network algorithm places these locations in the center.  City nodes are scaled by the number of NLJ 250 HQ's in the city.  New York has the most (35), followed by Chicago (19) and Washington (17).  Why is London in the center?  Because firms with offices in London overwhelming have offices in NYC and Washington.  For some contrast, I added labels for some outer-periphery cities.
• Firms.  Larger firms, depicted by larger circles, are clustered in the space between or around the important cities.  In other words, large firms are integrally linked to large cities--very large firms all have a presence in most or all of the key cities.  The outer periphery represents firms with more distinctive networks of less important cities.  The farther apart the circles, the more dissimilar the firms' networks.

There is nothing truly new or novel in this graph (those will come later), though it does suggest that large firm practice in a larger city probably entails substantively different (probably more specialized) work than large firm practice in smaller markets.  A lawyer contemplating where to begin his or her career might contemplate the glass ceiling imposed by taking a job at an outer-periphery location or firm.  Among other modes of analysis, I will eventually generate 30 charts like the one above (from 1978 to 2007) to create an animated "flip book" showing how this constellation has changed over time.   

Kudos to Stan Wasserman, Bernice Pescosolido, and Ann McCranie for putting on a terrific ICPSR workshop.  Thanks to their class, I have five years worth of research ideas.

Update to Common Space Scores

An announcement from Chad Westerland, U of Arizona:

For those who might be interested, Lee Epstein, Andrew Martin, Jeff Segal, and I would like to announce an update to our Judicial Common Space ideology scores. These measures are an attempt to provide preference estimates for Supreme Court justices that are directly comparable to preference measures of Courts of Appeals judges, members of Congress, and the President. Our data extend to 2005 and now correspond with the most recent version of Keith Poole's Common Space scores. The scores can be downloaded along with the corresponding paper at:  http://epstein.law.northwestern.edu/research/JCS.html