Empirical Legal Studies

What is Empirical Legal Scholarship?

For many reasons, the work of empirical scholars has been gaining greater attention in law schools in recent years, right along with the work of other interdisciplinary scholars.   A few weeks ago during my guest blogging stint, I highlighted some of the challenges of the increasing focus on interdisciplinary scholarship in law schools, including how to treat co-authored work and the chasm between the work of legal scholars and scholars in other disciplines.

But after my series of posts, I began asking myself the fundamental question of whether empirical work is necessarily interdisciplinary in nature?  To be sure, in many cases, an advanced degree in another discipline can be an advantage in undertaking empirical scholarship.  But unlike law and [fill in the blank], empirical work is about methodology and applying statistical tools to understand or explain trends and other phenomena.  While many disciplines (such as sociology and political science) rightly emphasize statistical techniques and methodology, especially among its advanced degree students, no schools to my knowledge offer Ph.D.s in empirical methodology that are wholly divorced from an underlying substantive discipline (except maybe in statistics).  Why not?  It is because empirical techniques are not particularly useful without an understanding of the phenomena or trends that are being studied.

For that reason, I do not consider myself exclusively, or perhaps even primarily, an empirical scholar, even when I undertake work with an empirical focus.  In one of my current projects, for example, I have mined the papers of Justice Harry Blackmun in an attempt to determine the reasons why the Supreme Court's docket shrunk so precipitously between 1986 and 1993.  I largely have the answers now, but only part of the explanation can be found in the raw data itself.  Without an understanding of the history of the Court and the trends that were occurring at the time, I am not sure that the data alone would have been particularly useful to me.  In other words, it is difficult to form a workable hypothesis (absent letting the data entirely drive the process) without understanding the context of the issue or phenomenon being studied.  My ongoing project is as much about history and legal trends as it is about empirical methodology.

Thus, I posit the (perhaps too philosophical) question of whether empirical legal scholarship is just another form of legal scholarship--that is, is it just another way that we study legal doctrine and institutions?  Traditional doctrinal scholars use the language of opinions, statutory provisions, and other legal sources to argue for a particular understanding or construction of the law.   Empirical scholars simply use a different set of tools--statistical techniques and rigorous empirical methodology--to study the law and legal institutions.  Do we call empirical work interdisciplinary simply because many scholars who engage in such work also have advanced degrees in other disciplines or is there something more fundamentally interdisciplinary about our work that I am missing?

FURTHER THOUGHTS: By the way, I do not mean to undersell in this post the growing impact of empirical work on legal scholarship.  The growth of such scholarship over the past few years, in part to test the theories and hypotheses of other legal scholars, has been an important development in the academy and has no doubt fostered greater interdisciplinary collaboration.  I only ask this question to solicit thoughts about the nature of our work.

Eyewitness Identification

Stanley Fisher (BU) has written Eyewitness Identification Reform in Massachusetts.  The Abstract:

This article examines the impact of new scientific learning upon police eyewitness identification procedures in Massachusetts. Much has been written about the case for procedural reform, but little about how it can be accomplished in states with fragmented systems of local policing. We know even less about how reforms, once adopted, are implemented. This article explores both the factors leading to reform of eyewitness identification procedures in Massachusetts, and their implementation.

Between 1997 and 2004, local media publicized a number of prisoner exonerations in Massachusetts, many of which resulted from mistaken eyewitness identification. As a result, legislative bills were filed to require identification reforms, and pressures arose to establish a state "innocence commission" armed with subpoena power. In response, a number of district attorneys held trainings in reform procedures for police departments in their counties. This article examines obstacles to the spread of reform practices statewide, including: 1) the decentralized system of law enforcement, which allows each police department discretion to choose its methods of criminal investigation; and 2) the failure of police training authorities to enforce statewide training according to a uniform curriculum. Finally, the article describes a pilot study to learn how police in two "reform" counties have implemented reforms in photo array procedures. A review of 197 public defender felony case files revealed 53 in which police conducted and documented eyewitness identification procedures. The study suggests that 1) patterns of compliance and non-compliance exist; 2) most police failed to obtain statements of witness confidence in both photo arrays and field show-ups; 3) in one county, a problematic pattern might exist of conducting non-blind photo-arrays sequentially. In conclusion, the article proposes steps to expand adoption and improve implementation of reform procedures.

Do Blogs Influence SSRN Downloads?

Paul Ohm (Colorado Law) has posted a draft entitled Do Blogs Influence SSRN Downloads? Empirically Testing the  Volokh and Slashdot Effects.  The Abstract:

SSRN's download statistics are criticized for being biased in favor of bloggers. Just how does the supposed bias work, and how strong is it?

This paper reports the results of a small empirical study undertaken in April, 2007. While guest-blogging at the Volokh Conspiracy, the author used a small computer program to collect SSRN Abstract View and Download statistics every fifteen minutes.

The study took on an unexpected dimension when links to some of the author's blog posts appeared in an article on the Slashdot website, one of the most widely-read technology websites. This allowed the author to compare the Volokh Effect with the better known and more often studied Slashdot Effect.

This is a quickly-compiled draft summarizing and analyzing the results. There are many colorful graphs included. You should really click the Download link below. Seriously! Click it. You know you want to.

The odds are very good that the author is collecting data about this abstract page, as well.

Attorney Gender & Judicial Decision Making

I am currently working on a series of projects with John Szmer (University of North Carolina, Charlotte) and Tammy Sarver (Benedictine University) concerning the influence of attorney gender on judicial decision making. In a paper recently posted on the ELS Blog, "HAVE WE COME A LONG WAY, BABY: FEMALE ATTORNEYS BEFORE THE UNITED STATES SUPREME COURT," we tested a “gender schema” hypothesis by examining the influence of the gender of the orally arguing attorney on the likelihood of the petitioner receiving a favorable vote. We also tested a “different voice” hypothesis via the gender composition of the litigation team. When we examine the gender of the orally arguing attorney and the gender composition of the litigation team in separate models, we find a negative and statistically significant relationship between the attorney gender variables and the vote. However, when included in the context of the same model, the negative relationship between the orally arguing attorney’s gender and the likelihood of receiving a favorable vote is no longer statistically significant. The litigation team gender variable, however, retains its negative relationship and is statistically significant.  This led us to reject the idea that the application of negative gender schema was at work, since this should most clearly manifest itself with respect to the orally arguing attorney. Instead, we conclude that the results tend to support the “different voice” theory.

In working on the project, we agreed that it would be interesting to test whether the relationship between attorney gender and the vote was conditional upon the justice’s gender and, indeed, we do report these results (in the text). With only two female justices on the Court, however, our ability to generalize is limited. Most recently, we have analyzed a similar model in Supreme Court of Canada (SCC) decisions. In the time period of our Canadian study, four women sat as justices on the SCC. The current Chief Justice, moreover, is a woman. In our preliminary analysis, we found that petitioners who employ a larger number of women on their litigation teams relative to the number employed by respondents are more likely to receive a favorable judicial vote. However, our results have not uncovered a conditional relationship between justice gender and attorney gender. Moreover, although we made strong attempts at building comparable models, we were not able to discern who orally argued the case before the SCC. Consequently, our results in the Canadian context were limited to an analysis of the gender composition of the litigation team.

Forgotten Racial Equality: Implicit Bias, Decision-Making and Misremembering

Justin Levinson (Hawaii - Richardson) has written Forgotten Racial Equality: Implicit Bias, Decision-Making and  Misremembering.  The Abstract:

This Article argues that judges and jurors unknowingly propagate racism through their legal decisions because they misremember case facts in implicitly biased ways. Such an argument bridges discourse in implicit social cognition, memory studies, and legal decision-making. Social cognition research indicates that implicit racial biases are automatic, pervasive, and often operate without conscious awareness. Memory studies demonstrate that memory errors are meaningful and systematic, and are often facilitated by stereotypes. Decision-making theory teaches that memory errors can not only affect the results of individual decision-making processes, but also that group deliberations are unlikely to resolve these errors. The author argues that these three areas converge, resulting in a model that explains how implicit memory errors are made unconsciously and in racially biased ways. These unconscious and pervasive implicit memories biases consequently have the ability to affect a variety of legal outcomes.

To test the theory, the author conducted an empirical study that examined how people remember legally relevant facts. In the study, participants read two short stories resembling legal cases - one about a fight and another about an employment termination.

Judging the Voting Rights Act

Cox and Miles (Chicago) have written Judging the Voting Rights Act.  The Abstract:

The Voting Rights Act has radically altered the political status of minority voters and dramatically transformed the partisan structure of American politics. Given the political and racial salience of cases brought under the Act, it is surprising that the growing literature on the effects of a judge's ideology and race on judicial decisionmaking has overlooked these cases. This Article provides the first systematic evidence that judicial ideology and race are closely related to findings of liability in voting rights cases.

Democratic appointees are significantly more likely than Republican appointees to vote for liability under Section 2 of the Voting Rights Act. These partisan effects become even more prominent when judges appointed by the same president sit together on panels. Moreover, a judge's race appears to have an even greater effect on the likelihood of her voting in favor of minority plaintiffs than does her political affiliation: minority judges are more than twice as likely to favor liability. This finding contrasts starkly with prior studies of judicial decisionmaking - studies finding that, across a range of legal questions, a judge's race has only a weak effect, if any, on the resolution of cases. As with partisanship, the so-called "panel effects" of race are strong, as white judges become substantially more likely to vote in favor of liability when they sit with minority judges. These findings have significant implications for a number of controversies, including debates about which institutions are best situated to protect minority voting rights and disputes about the role of diversity within the federal judiciary.

Why Trial Court Judges Write Opinions Rather Than Orders

David Hoffman et al. (Temple, Law) recently circulated a paper that will interest those who analyze judicial opinions.  Their paper, Docketology, District Courts, and Doctrine, challenges assumed motivations regarding a judge's decision to write and publish an opinion as well as work that relies on written judicial opinions as a window into judicial behavior.  An excerpted abstract follows.

"Empirical legal scholars have traditionally modeled judicial opinion writing by assuming that judges act rationally, seeking to maximize their influence by writing opinions in politically important cases. Support for this hypothesis has reviewed published opinions, finding that civil rights and other “hot” topics are more to be discussed than other issues. This orthodoxy comforts consumers of legal opinions, because it suggests that opinions are largely representative of judicial work.

***

Using a hierarchical linear model, our statistical analysis rejects the conventional orthodoxy: judges do not write opinions to curry favor with the public or with powerful audiences, nor do they write more when they are younger, seeking to advance their careers. Instead, judges write more opinions at procedural moments (like summary judgment) when appeal is likely and less opinions at procedural moments (like discovery) when it is not. Judges also write more in cases that are later appealed. This suggests that the dataset of opinions from the trial courts is significantly warped by procedure and risk aversion: we can not look at opinions to capture what the “Law” is."

State Dominance of a Circuit

Thank you for the opportunity to participate as a guest blogger this week!  In my first post, I would like to share some thoughts on a paper I discussed at the recent Midwest Political Science Association meeting in Chicago. The paper, Prof. Stephen Wasby’s “State Dominance of a Circuit: An Exploration,” (available on the conference’s archive at http://64.112.226.77/one/mpsa/mpsa07/index.php?cmd=mpsa07&id= ) provides an interesting comparative analysis of state and district dominance of circuit court law.  I suspect, moreover, that it might be of interest to some who regularly review the ELS Blog.

The paper is motivated by discussions concerning the possible split of the Ninth Circuit and the noted dominance of California in Ninth Circuit litigation.  Prof. Wasby provides an exploratory analysis of this dominance across multiple measures of the concept including the number of appellate filings originating from the state as well as its share of published and unpublished decisions, en banc rulings, and Supreme Court rulings.  What I found particularly interesting was the comparative approach employed; California’s dominance in the Ninth Circuit was compared to that of other states in other circuits including New York in the Second Circuit, Pennsylvania in the Third Circuit, Texas in the Fifth Circuit, Illinois in the Seventh Circuit, and Florida in the Eleventh Circuit.  Although some of the data across the above measures was limited to the Ninth Circuit, the descriptive study helped to place the issue of California’s dominance in perspective.  The paper also included a section on district court dominance as well, and it too was comparative in nature.

The paper, however, also raised a number of interesting research questions.  Implications concerning future divisions of the Ninth aside (which are important enough to justify taking an empirical look here with respect to California), as someone interested in analyzing the influence of circuit court law on circuit judge decisions, it does encourage one to think more deeply about the development of that law.  Consequently, this work can provide a point of departure for further analysis of the substantive implications of the fact that certain districts and states within a circuit contribute differentially to the cases that form the basis of circuit law.

Guest Blogger: Erin Kaheny

This week's guest blogger is Erin Kaheny, Assistant Professor of Political Science at Wisconsin-Milwaukee.  Her research interests include federal appellate court decision making, procedural rules of access to the legal system, judicial careers and acclimation effects, divided government and judicial selection, agenda change in the U.S. Courts of Appeals, and comparative judicial studies.  Recently, her paper draft entitled Have We Come a Long Way Baby: Female Attorneys before the United States Supreme Court has garnered much attention by both blogs and the media (perhaps she'll discuss this).

New Book on Justice Thomas

Supreme Discomfort: The Divided Soul of Clarence Thomas
by Kevin Merida and Michael Fletcher (Doubleday 2007)

Availability: Amazon / B&N

Review: Kenji Yoshino in the Washington Post

Via Legal History Blog

Sources and Books for Empirical Research

As you may recall, Bill Henderson and I had a couple of posts expounding the virtues of the Sage monograph series on "Quantitative Applications in the Social Sciences" several weeks back.  It is a great starter series (though expensive for the whole thing) for anyone looking to enter empirical research with or without a quantitative background.  The library here at Minnesota has asked me to help build the collection of empirical and quantitative resource materials, and I am wondering whether anyone has any suggestions about "must have" books in the area?  I have already recommended that the library order a number of selections, including from the Sage Series, but I don't want to miss any important books in the field.  Your comments, as always, are most welcome.

The New Economics of Survey Research

I am a member of the Indiana State Bar Association and part of the organizing committee for the Annual Solo & Small Firm Conference.   I got involved with this effort several years ago when I attended the annual conference in preparation for my Law Firms course.  It is a remarkable group of professionals who have taught me a lot about the many soft and hard skills necessary to build a successful and satisfying law practice.   Therefore, when the group asked me to conduct a member survey of financial and billing practices, I was happy to reciprocate.

But here is the hitch:  we did not have much of a budget.   Remarkably, thanks to a wonderful web utility called SurveyMonkey.com, lack of funding is becoming less of an obstacle for survey research.   Here is a copy of my 35 question survey to ISBA members, which was approved by Indiana University's IRB committee (as a side note, my total time allocation to get IRB approval was eight hours--far from a horror story).   Responses to the survey were generated via an email from the ISBA Executive Director that solicited member participation.  That email contained a link to the Study Information Sheet (an IRB requirement), which was hosted on the ISBA website.  A link to the actual survey was placed at the bottom of the Study Information Sheet.

For a whopping $360 (for a full-year premium service subscription) payable to SurveyMonkey.com, I got the following:

• A wonderful survey tool with easy-to-use drop down menus;
• A secure, encrypted Internet connection for each survey respondent (without this option, the price drops to $240);
• The ability to view preliminary results in real time;
• A large digital file of results that could be readily imported into Excel, SSPS, Stata, or another statistical program;
• 1,280 respondents.

Frankly, it is too early to tell whether our solicitation method produced a biased sample.  (And if so, we may be able to fix that through a weighting methodology based on what we know of the whole ISBA population.)  But the preliminary results are certainly encouraging.  The new economics of survey research made this possible.    FYI, if I used the Indiana University Survey Center to conduct this poll, the total cost would have been in excess of $8,000. 

New Contract Law Collection

Economics Of Contract Law
Edited by Douglas G. Baird (Edward Elgar 2007)

At $275.00 retail, this collection of previously published articles is not exactly priced for the individual book buyer, but check out the library's copy for Baird's introduction to the topic and his selection of greatest hits.

The publisher's listing is here. The table of contents is after the break.

Thank You

I would just like to thank Bill and the rest of the ELS folks for the kind invitation to join the blog as a permanent editor.  I look forward in the coming weeks and months to posing questions similar to those that I have advanced over the past two weeks, to cross-post to SCOTUSblog.com occasionally on Supreme Court-related issues , and to ask your advice on my empirical projects and those of my colleagues and students in both the political science department and law school here at the University of Minnesota.  I also would love to hear about your experiences in conducting empirical research, and in that regard, please feel free to e-mail me if there are any issues that you would like for me to raise on this blog.  I anticipate that, through even informal collaboration on this blog, we will all turn out better quality empirical work in the future.

David Stras Joins the ELS Blog

The Empirical Legal Studies Blog is pleased to announce the addition of David Stras as a permanent editor.  David is an Associate Professor of University of Minnesota Law School and teaches courses in criminal law, federal courts, constitutional law, and law & economics.  He holds B.A, M.B.A., and J.D. degrees from the University of Kansas, where he served as Editor-in-Chief of the Criminal Procedure Edition of the Kansas Law Review. 

David is empiricist who leverages his institutional knowledge and work experience to ask new and relevant questions on the deliberations and functioning of the federal courts.  David has served as a law clerk to The Honorable Melvin Brunetti (9th Circuit), The Honorable J. Michael Luttig (4th Circuit), and Justice Honorable Clarence Thomas of the Supreme Court of the United States.  In addition, he worked as an appellate lawyer for Sidley Austin Brown & Wood.    Dave is also a regular contributor to the SCOTUSBlog and editor of the SSRN Law & Politics Series.

We look forward to David's thoughtful and informative posts.  David, welcome aboard!

Gendered Pay or Choices?

Joni Hersch (Vanderbilt Law & Econ) recently circulated an intriguing paper assessing whether gender discrimination can be dismissed as an explanation for pay disparity between men and women.  Indeed, the title of her paper, Sex Discrimination in the Labor Market, gives away the thrust of her argument.  An excerpt from the abstract follows.

"The main issue considered in this paper is whether gender differences in choices, especially with respect to the family and household, are indeed responsible for the gender pay gap, or whether discrimination plays a role. On balance, the evidence indicates that sex discrimination remains a possible explanation of the unexplained gender pay gap. This is consistent with the continuing high profile sex discrimination litigation suggestive of on-going inferior treatment on the basis of sex."

Law & Social Science: A Comparative Approach

In the last few weeks, we have listed some very attractive opportunities for ELS-related education opportunities over the summer.  Here is another one: 

Cornell Law School and the Universite Paris I Pantheon-Sorbonne are cosponsoring their fourteenth annual Paris Summer Institute, a five-week academic program in international and comparative law.  The course are ABA-accredited and taught in English.  This year there are a dozen course offerings, including "Law & Social Science: A Comparative Approach," which will be taught by Valerie Hans (Cornell Law). 

The course will examine how social science expertise is employed in legal cases in multiple countries.  It will also address how common law and civil traditions, the diversity of legal procedures, and the range of legal cultures affect the use and impact of social science in law.

Thank You and Goodbye

Although I have not had much time to post this week, I want to thank Jason and the rest of the ELS crew for a wonderfully productive and interesting experience over the past two weeks.  Many thanks also go to those who took the time to read and make thoughtful comments to my posts.

ECPR Summer School in Methods and Techniques

Rebecca Morton (NYU Dept. of Politics) has asked us to post the following announcement, which appears to be European counterpart to the ICPSR Summer Program:

The European Consortium for Political Research (ECPR) is pleased to organize the second edition of the ECPR Summer School in Methods and Techniques, in cooperation with the University of Ljubljana (Slovenia), from 19 July to 4 August 2007. It is targeted at advanced students (Master, beginning Ph.D) and junior researchers in political science and sociology, but students and researchers from all social and behavioural sciences are most welcome.

The aim of the summer school is to provide methods training opportunities in addition to MA and PhD curricula at European universities and to foster debate, interaction, and mutual understanding between various methodological perspectives at an early stage in academic careers. The curriculum covers introductory and intermediate courses in both quantitative and qualitative methods, also stemming from different ontological and methodological perspectives, and also features some plenary interactive events and debates, so as to increase alertness of young researchers to other methodological perspectives and possibilities. Some ECTS credits (3 or 5) can also be earned for the main, two-week courses.

The programme has been expanded to 4 intensive courses and 15 main, two-week courses.  The 4 Intensive courses (three days: 19 July - 21 July 2007) are:  1. Inferential statistics; 2. Basic concepts of mathematics; 3. Introduction to SPSS; 4. Introduction to R.

The 15 Main courses (two weeks: 23 July - 4 August 2007) are:  5. Mathematical concepts and formal modelling;; 6. Generating 'qualitative' data: expert interviews and documentary sources;  7. Comparative survey design;  8. The statistical analysis of cross-national survey data;  9. Multiple regression analysis;  10. Political and policy ethnography;  11. Quantitative narrative analysis;  12. Comparative research design and configurational comparative methods;  13. Network analysis;  14. Qualitatively orientated textual analysis;  15. Online research methods;  16. Temporality and sequence analysis;  17. Mixed methods design;  18. Multilevel analysis ; 19. Methodologies of case studies.

Full information is available on the official Summer school website.  Registration to each course is accepted on "first come, first serve" basis.

The Underutilization of Descriptive and Basic Inferential Statistics?

I would like to thank the readers of ELS for the excellent comments thus far to my previous posts.  In this post, I want to raise an issue that I have been thinking about for quite some time.  Although not every project or issue can be appropriately analyzed using descriptive or basic inferential statistics, it seems to me that many empirical scholars (both within the legal academy and outside of it) underestimate the value of using basic statistics to explain phenomena or relationships between variables.  Instead, empirical scholars often look for the most complex models, which result in papers that are impenetrable for non-quantitative scholars, rather than searching for ways to present the data in a fashion that can be understood by a wider audience. 

As someone who sits on dissertation committees in other disciplines on occasion and tries to attend workshops in political science as much as possible, I notice this trend among graduate students as well: in a zest to conform to the norms of their discipline, graduate students will sometimes forget that there can be enormous (and sometimes more!) value to presenting the data using basic statistical techniques.  If we, as empirical scholars, want to reach a wider audience for our work, is it worthwhile to simplify the empirical papers that we write when possible?  Is reaching a wider audience even a worthwhile goal?  Would courts and policymakers pay more attention to our scholarship if they could understand it?  Again, I realize that not every project lends itself to simple statistical techniques, but I thought these questions worth asking nonetheless.  As always, your comments are most welcome.

Special Oxford University Spring School course, June 18-20

Chris Zorn has asked us to post the following announcement:

The Department of Politics and International Relations at Oxford University will be offering a special Spring School course, June 18-20, 2007.  The title of the course is "An Introduction to Event History Analysis," and the course will be immediately preceded by a "Crash Course in Stata."

The deadline for registration is May 18, 2007.  Details on the course can be found at here.  Note that the registration fees vary slightly as a function of whether or not a participant's institution is an ECPR member.  Many U.S. institutions are; check with members of your political science department if you're not sure.

Seems like a nice way to combine business with pleasure.

ELS Listserv

Thanks to former guest blogger Carolyn Shapiro (Chicago Kent), the members of the newly created ELS Listserv are introducing themselves to each other.  There is a strong untenured tilt to the members, several who joined to facilitate an empirical project that draws on their substantive research fields, some really diverse work experience and educational backgrounds, and a surprising trans-continental representation.   Overall, a very interesting group.

If you want to get in on the ground floor, email Carolyn.

CORRECTION:  The listserv is untenured, pretenured, and graduate student by design.  When some of the members get tenure, the issue may be revisited.

Laptop Debate: Effect on Exams versus Class Discussion

Peter Lattman at the WSJ Law Blog is asking readers whether Professor David Cole's no laptop policy is a good idea.  Cole, who teaches at Georgetown Law, published an anti-laptop essay in the Saturday Washington Post.  Like Professor Cole's students, the comments appear to be favoring the ban.  But there are some dissenters.

Perhaps this is the knotty empirical question that lies beneath this controversy:  What if final exam performance is maximized by laptop transcription, but the quality of class discussion is maximized by no laptops?   I think there may be some truth to this.  Many students claim that laptop notetaking helps them better distill the information after class.  If so, it is quite a conundrum. 

Any thoughts on a research design to test it?

The Influence of Oral Arguments (With Update)

I would like to draw your attention to a paper, see here, that uses the oral argument grades assigned by Justice Harry Blackmun to examine various issues relating to the effectiveness and influence of Supreme Court oral arguments. In the paper, which is forthcoming in Washington University Law Review, Tim Johnson, Jim Spriggs, and Paul Wahlbeck examine a random sample of 539 cases decided by the Court between 1970 and 1994 (spanning Justice Blackmun's tenure).

The aspect of the article that is most interesting, at least in my opinion, is the part of the paper that discusses those attributes of oral advocates that are most influential to the oral argument grades assigned by Justice Blackmun. Using such variables as whether the solicitor general is personally arguing the case, the amount of experience for the advocate arguing before the Court, and whether the advocate is an elite Washington attorney, former Supreme Court law clerk, or law professor, the authors run an ordinary least square (OLS) regression to determine which of the aforementioned factors most influence the oral argument grades assigned by Justice Blackmun. The results are not surprising as the personal appearance of the Solicitor General generally leads to a higher grade, and the presence of a former Supreme Court clerk or elite Washington attorney are also influential on the quality of an oral argument.

I do, however, question the grader in this case. Justice Blackmun appears to have been an especially harsh grader and the grades he assigned to several prominent legal figures are quite surprising. In the case of now-Chief Justice Roberts, who I believe is one of the great oral advocates of the past two decades, Justice Blackmun gave him only "average" grades in the twelve cases in the sample. In fact, in one of them, he said that Roberts did "better than usual," earning a grade of six on an eight point scale. When I clerked at the Court, I always made a point of being in the courtroom for Roberts' oral arguments as I always felt that there was much to be learned from his exceptional advocacy style and level of preparation. In addition, Blackmun gave only average grades to Justice Alito, and Justice Ginsburg did not fare particularly well in the sample either.

Note: Cross-posted at SCOTUSblog.com.  Please also feel free to comment on my post on the challenges of data coding below, and I hope to have a post completed early next week on the use of descriptive statistics in empirical scholarship.  I would like to thank Jason for permitting me to hang around for another week.

UPDATE: In response to my cross-post on SCOTUSblog.com yesterday, Howard Bashman wrote an "On Appeal" column for Law.com discussing the paper and explaining the importance of having experienced appellate attorneys arguing before the Court, see here.

Is there a Coastal Bias in the Lawyer/Judge US News Survey?

Over at the Legal Professions Blog, my good friend Jeff Lipshaw, equipped with an excel spreadsheet, a map, and a ruler, has taken on the perennial claim that the Lawyer/Judge input variable in U.S. News contains a "coastal bias."   His analysis says no, but Jeff has asked for some help. 

Using a simple OLS model, my dependent variable is 2006 Lawyer/Judge Reputation.  My independent variables include Academic Reputation, 75th percentile LSAT (which is good proxy for total rank), a west coast dummy (Arizona, California, Oregon, and Washington; 27 schools = 1) and an east coast dummy (Delaware, Maine, Maryland, Massachusetts, New York, New Hampshire, New Jersey, Pennsylvania, Rhode Island, Vermont, and Virginia; 54 schools = 1) .  Here is a summary of the results, which corroborate Jeff's theory:

Obviously, the signs for both coasts are negative (i.e., no systematic boost for either coast), and the p-value for East Coast implies a statistically significant penalty for being an east coast school.  Perhaps schools far away from the major east coast legal markets are destine to seem better than the nearby law schools that firms know well but tend to run down.   The adjusted R-squared for the model is 75.7%.

We might be able to specify a better regression model, or recode with a better definition of "coastal", or correct for some minor heteroscedasticity.  But anyone who thinks they can salvage the bias theory is probably kidding themselves. 

Jeff, you should plan on spending the summer in Ann Arbor at the ICPSR statistics boot camp.  You clearly show promise. 

Dummy Variables in Cross-Country Regressions

I post the following at the request of a current student at Marquette.  If you have any insights for him, please post a comment or email me at [email protected].

"I am writing a paper on economic growth and want to look at the effects, if any, that the mixture of civil law and common law in Louisiana has had on that state's rate of growth.  Theory and past research using cross-country regressions show that nations with civil law origins have lower rates of economic growth, all else constant.  I want to use a cross-state comparison (LA, MS, AL, and AR) and include a variable that captures the affects of LA's mixed legal system of the state's rate of growth.  Most of the research I have looked at uses a simple binary choice dummy variable to distinguish the legal origin.  However, as you probably know, this dummy can pick up influence from any number of variables not included in the model and that are unrelated to legal origin.  Therefore, I have been trying to come up with some way to capture the legal origin distinction, but with a proxy variable that provides a greater quantitative distinction for legal origin than the dummy.  Some quantifiable measure that would theoretically move with the ebb and flow of civil law influence on the state's legal system, which has waxed and waned since LA became a state.  There may be no real good solution for this problem (which I imagine is why dummy variables exist in the first place), but I would appreciate greatly any helpful comments or ideas." 

The Challenges of Data Coding

Many of my recent projects have required quite a bit of data coding.  My review essay in this month's issue of the Texas Law Review, for instance, required the review of approximately 20,000 pool memos from the Blackmun papers in order to evaluate the characteristics and possible influence of the Supreme Court's cert pool.  Another project, which I hope to co-author with Tim Johnson (University of Minnesota Political Science Department), will examine the factors that influence the certiorari votes of each of the Justices on the Court during the latter part of Justice Blackmun's tenure on the Supreme Court (1986-1993).  Finally, I have been collecting data for almost a year for an article examining the reasons behind the Supreme Court's declining plenary docket.  Among other things, the latter two papers have required coding the cert votes of every Supreme Court Justice in every case (with at least one vote to grant or join-3) from 1986-1993.   The data set includes between 2,000-3,000 cases, and approximately 20,000 total votes.

The amount of data collection required for these various projects is extensive, and I can now say that the collection and coding of the data has been one of the most difficult aspects of empirical research.  The first challenge, of course, is figuring out exactly how you want to code the data, and which variables should be included in the data set.  Without careful consideration, you may quickly find yourself well into the project, and having to review and reenter much of the data because an important variable has been unintentionally omitted.  I have found that, when examining and studying courts, the importance of some variables is often not self-evident, and thus it pays to be overinclusive in the amount of information and number of variables coded on the front end.   

The second challenge is competence.  Law students, unlike graduate students in the social sciences, are not trained in data coding. Very few have quantitative backgrounds, and even fewer have experience working with large data sets and statistical packages such as SPSS and Stata.  As a result, my empirical research has required me to train and review the work of my research assistants quite closely, which takes time away from the research itself.  Thankfully, I now have a research assistant that has become quite proficient at coding and interpreting data, but I am coming to grips with the fact that he graduates at the end of this year and I have yet to find a suitable replacement.  Again, unlike graduate students, law students are barred from working for professors during their first year, and thus it is only possible to work with them for a maximum of two years, hardly enough time for them to gain competence and speed in data coding.  One possibility, of course, is to hire graduate students in the social sciences, but for some reason beyond my understanding, it is far more costly to hire research assistants from outside the law school (as much as three times as expensive).   I still have yet to find a suitable answer to the second issue, and I wonder how my colleagues in the legal academy deal with this question.  Comments to this post are therefore most welcome.     

Interdisciplinary Scholarship and Co-Authorship

I noted yesterday that law schools have really focused on interdisciplinary scholarship in recent years, especially when hiring new faculty.  But it is also fascinating that many law schools have failed to adjust to the type of scholarship produced by such scholars.  Many newly-hired faculty have Ph.D.s in such areas as economics, history, or political science, and are accustomed to the promotion and publishing standards of the disciplines in which they did the majority of their post-graduate work, not those of the legal academy. 

For example, my experience in political science is that it is rare to see articles that are produced by only a single author.  Some papers have five co-authors or more, and most have at least two.  Yet my conversations with interdisciplinary scholars at many law schools seem to indicate that co-authored works are discounted, especially when more than two authors are listed on the piece.  Another prominent example are books, which often take far longer to produce than an article, yet again many law schools do not give such works the credit that they deserve. Yet as we move into a new era of interdisciplinary research, shouldn't our standards adjust to compensate for the new focus? 

As a young scholar, I have passed on or postponed a number of interdisciplinary collaborations due to my own uncertainty about how the works produced would be viewed both within and outside of my institution.  Yet I also know that collaborations with my colleagues in political science or history would result in works that are better and more insightful than what I can accomplish on my own.  My own experience with co-authorship is that each of the authors puts in about 75% of the work of a solo-authored project, primarily because collaboration results in many discussions about the work with your co-author, and nearly all of the editing and thinking that would be required of a solo piece.  If properly split up, the only aspect of co-authorship that is easier is that you do not have to all of the research and writing of the piece; instead, that obligation is shared by both authors.  I greatly enjoy and have done both types of projects, solo and co-authored, but the institutional incentives seem to clearly favor solo pieces even though the legal academy has made this steady march toward interdisciplinary work.

Tomorrow, I hope to have a post commenting on the challenges of coding empirical data.          

Law and Politics

My post this morning was designed to consider the challenges posed by increasing interdisciplinary scholarship in law schools aside from the traditional critique that legal scholarship is no longer useful for judges and practitioners.  The rise in the emphasis by law schools on interdisciplinarity has thankfully been accompanied by an increase in resources available for the interdisciplinary scholar. 

One valuable resource is this blog and others like it.  ELS, for example, brings together scholars from a variety of disciplines, including political science and the law.  As someone who has this site bookmarked and reads it relatively often, I find papers posted here that would otherwise not cross my radar screen. 

Another is SSRN, which as you know serves as a repository for papers from a variety of disciplines.  In an effort to promote greater awareness of scholarship in the areas of political science and the law, Tim Johnson (University of Minnesota Political Science Department) and I have started an SSRN e-journal on "Politics and the Law," which is now on the Legal Scholarship Network and will be cross-listed on the Political Science Network once it is created.  If you have interest in the intersection between politics and the law, I encourage you to subscribe to this newly-created journal.  Similarly, other disciplines, such as legal history, have created e-journals for much the same purpose.  The Experimental and Empirical Studies e-journal is likewise excellent for empirical scholarship.

Finally, as I mentioned this morning, conferences promoting interdisciplinary scholarship are becoming more widespread.  A great example is the Empirical Legal Studies Conference, which features papers by scholars from a variety of disciplines.  Although I had a conflict with the conference last year, it is my hope to at least attend the conference this fall and possibly even submit a paper for consideration.  The line-up of presenters last year was most impressive, and I expect that it will be equally, if not more, remarkable this fall.

I am sure that a number of other resources exist that I am unintentionally omitting, and would welcome other suggestions in the comments to this post.  In a post tomorrow, I hope to continue the discussion by considering the relationship between interdisciplinary scholarship and co-authorship, and whether interdisciplinary scholars may be at a disadvantage due to law schools' traditional discounting of co-authored works. 

Interdisciplinary Legal Scholarship

In recent years, interdisciplinary legal scholarship has become more common, and many law schools seem to be placing greater emphasis on recruiting scholars with an interdisciplinary bent.  Although I am by no means familiar with many of the "law and [insert field]" areas of scholarship, my own research on the Supreme Court requires familiarity and facility with political science scholarship on judicial decision-making.  Although the environment seems to be slowly improving with a greater number of political scientists publishing in legal journals, there still seems to be a disconnect between the work of legal scholars and political scientists, with many in both fields ignoring the scholarship of the other.  I have often wondered about the reasons for this phenomenon.

One simple possibility is that political scientists and legal scholars have historically published in journals that are housed in different electronic databases (or in the pre-computer days, different libraries).  For instance, most political science journals are not available on WESTLAW or LEXIS, but can be found on such electronic databases as JSTOR and Academic Search Premier.  Of course, in law school and private practice, lawyers are taught to rely heavily on WESTLAW and LEXIS, but have little familiarity with the other electronic databases.  Thus, a "quick search" on WESTLAW will not reveal "on point" scholarship from other disciplines.

Another possibility is the insularity of each of the disciplines.  For many years, the work of legal scholars consisted primarily of doctrinal work and normative recommendations stemming from theoretical (rather than empirical) policy arguments.  Political scientists have often viewed the work of legal scholars as insufficiently methodologically rigorous, even when we have ventured into empirical research.  To some degree, that might even be true today, although the amount and degree of collaboration between political scientists and legal scholars has vastly improved.  Meanwhile, legal scholars have criticized political scientists for failing to see the nuances in the law, and focusing too much attention on institutional and external factors to explain judicial decisions.  Legal scholars also criticize political scientists for rigid adherence to models of judical decisionmaking without explaining the outliers in the data or the imperfections of the models.    

I am sure that there are other possibilities, and I don't mean this post as a criticism of either discipline, but I do believe that greater collaboration between political scientists and legal scholars will result in more complete explanations of judicial behavior.  A legal scholar that focuses solely on doctrine may be missing half the story, while a political scientist that ignores the law might be missing the other half.  Having studied the Supreme Court and other federal courts from several different perspectives (including from the inside), it seems clear to me that judicial decisionmaking is a product of a host of factors, some of which political scientists are better able to evaluate (such as external political constraints) and others that are more within the expertise of legal scholars (such as stare decisis and nuances in doctrine).

I see it as a positive sign that political scientists now participate widely in legal symposia, and legal scholars now present papers at political science conferences.  I also know that many of my colleagues now run a search (as I do) in Academic Search Premier or JSTOR before writing an article on the court system or judicial decisionmaking.  In addition, I can say without hesitation that I have personally learned a great deal from discussions with my political science colleagues and from attendance at presentations sponsored by Minnesota's political science department.  The next step, I think, is to see greater co-authorship between legal scholars and political scientists.  Even so, I see a healthy trend developing.   

More Thoughts on the Sage Series

I would like to first thank Jason and the rest of the ELS crew for inviting me as a guest blogger this week.   I hope to get my first substantive post up tomorrow about my thoughts on interdisciplinary legal scholarship, but I would first like to second Bill's observation about the quality and helpfulness of Sage's Quantitative Applications in the Social Sciences series.  For novice researchers trying to understand basic concepts relating to both linear and multivariate regression, I  recommend Larry Schroeder's Understanding Regression Analysis monograph.   For a slightly more advanced examination of regression, Michael Lewis Beck's Applied Regression monograph is also a wonderful resource.  A number of the other papers  deal with the nuances of descriptive statistics, such as Peter Chen's Correlation and Michael Lewis Beck's Data Analysis monographs. 

Although there are no doubt a number of useful resources for empirical researchers, I find the  offerings in the Sage series to be among the best and most diverse.  A researcher that has questions about confidence intervals, for example, can turn to a Sage monograph focused solely on that subject.  I have encouraged our librarians at the University of Minnesota to purchase a number of the offerings from Sage as I think they are an indispensable (and perhaps underutilized) tool for methodologically rigorous empirical scholarship.

Basic Math for Social Scientists

Thanks to feedback received from my earlier post on Survey Research, I purchased a few more books from the Sage Quantitative Applications in the Social Sciences series:  Arlene Fink, The Survey Kit (2002) and Graham Kalton, Introduction to Survey Sampling (1983).  These monographs are short and readable explanations for people interested in specific research methodologies or background theory.

While purchasing these books, I also bought another Sage monograph, Tim Hagle, Basic Math for Social Scientists: Concepts (1996), which arrived this morning.  As I have gotten deeper into empirical work, I really want to fully understand the math that is driving the Stata and SPSS regression models and diagnostic tools.

Fortunately, this little book by Hagle (who is a lawyer and a political scientist) turns out to be the book that I was one day hoping to find: a quick, nuts & bolts discussion of the important math concepts that underlie probability and multivariate regression:  algebra review of exponents and logarithms, limits and continuity, differential and integral calculus (which I have forgotten from college), matrix algebra, and eigenvalues and eigenvectors.  It also has a handy summary of math symbols and expressions. I am very happy with this purchase.

The 2007-08 Hylton Law School Rankings

Last year at this time, I posted the first incarnation of the Hylton Law School Rankings. "The Hylton Rankings rank American law schools in an order which reflects the way that they are regarded by law school professors and students. The Rankings are named for their compiler, Professor J. Gordon Hylton of Marquette University.... Unlike the US News ratings, the Hylton Rankings list law schools from #1 to #184."

Click here to view the 2007-08 Hylton Rankings

Guest Blogger: David Stras

This week's guest blogger is David Stras, Associate Professor of Law at the University of Minnesota.  Professor Stras teaches and writes in the areas of criminal law, federal courts and jurisdiction, constitutional law, and law and economics. His current research focuses on the federal judiciary and the Supreme Court of the United States.  Using a variety of methodological tools, including empirical and historical analyses, Professor Stras's research has examined a variety of issues relating to the Supreme Court.  Professor Stras is a frequent television and radio commentator on issues relating to the federal judiciary, particularly with respect to the Supreme Court of the United States.  He is also an adjunct faculty member of the University of Minnesota Political Science Department, and is the co-editor of a new SSRN journal on "Law and Politics." 

Professor Stras received his B.A. degree, with highest distinction, and his MBA from the University of Kansas. He received his law degree from the University of Kansas School of Law in 1999, where he served as Editor-in-Chief of the Criminal Procedure Edition of the Kansas Law Review. Following law school, Professor Stras clerked for The Honorable Melvin Brunetti of the United States Court of Appeals for the Ninth Circuit and then for The Honorable J. Michael Luttig of the United States Court of Appeals for the Fourth Circuit. From 2001 to 2002, he practiced white-collar criminal and appellate litigation with the Washington, D.C. office of Sidley Austin Brown & Wood. Following his year in practice, he clerked for The Honorable Clarence Thomas of the Supreme Court of the United States.

In addition, he a frequent and regular contributor to SCOTUSblog.