Empirical Legal Studies

Thanks for the welcome

I've been a regular reader of the blog since its inception.  I don't know how much actual posting I can contribute -- I have found some of the greatest value from the blog comes from our guest commentators, such as the current discussion of statistical significance. 

While I have for some time agreed with the commentators' contentions, I would add one perspective from an institutional/legal approach.  Any decision rule, be it a law or regulation or standard for evaluating research is going to be inaccurate.  It can't take into account the diversity of circumstances to which the rule would be applied.  And the reason for having such a rule is that it will still be more accurate than the free-for-all of unrestricted case-by-case evaluation.  While the statistical significance rule is substantially flawed, what would be better?  Professor Henkel has provided valuable explanations of how the standard is misused inferentially, but I wonder if there is not a better standard (rule) that could be deployed for research.  I think the overemphasis on statistical significance will continue until a better rule is designed.

Welcome to New Permanent ELS Blog Editor: Frank Cross

On behalf of the ELS Blog Editors, it gives me great pleasure to welcome Frank Cross as a new permanent Editor.   As many ELS folks already know, Frank is a leading empirical legal scholar who holds appointments in Texas' law and business schools.  Frank's scholarship traverses several fields, including the economics of law and litigation, administrative law, and environmental law. His descriptive and normative studies of judicial decision-making have received particular attention for their important insights, creativity, and methodological rigor.  A brief snippet of Frank's representative publications illustrates his scholarly breadth and productivity.  Since 1998, he has published more than twenty articles, including "What's Not to Like (About Being a Lawyer)?" (Yale Law Journal, 2000) (with C. Silver), "Institutions and Enforcement of the Bill of Rights" (Cornell Law Review, 2000), "Realism About Federalism" (New York University Law Review, 1999), "A Modest Proposal for Improving American Justice" (Columbia Law Review, 1999) (with E. Tiller), "Shattering the Fragile Case for Judicial Review of Administrative Rulemaking" (Virginia Law Review, 1999), and "Judicial Partisanship and Obedience to Legal Doctrine" (Yale Law Journal, 1998) (with E. Tiller).  As well, the ELS Blog Editors noted that Frank's contributions to this blog (and others) have been unfailingly constructive, collegial, and probative.

As the ELS Blog approaches its 100,000 site visitor, the Editors felt it an appropriate moment to improve what we hope readers and participants conclude is a helpful and worthwhile endeavor. The addition of Frank as an Editor accomplishes just that.

The social science controversy over significance testing

The social science controversy over significance testing arises from two, often overlapping, sources so there is arbitrariness in saying a criticism of significance testing stems from either source. One source concerns the technical requirements of significance tests, the second source is essentially a philosophy of science question, whether significance tests provide information of value in developing and validating scientific theories (call this process scientific inference). There are those who feel that in social science research, one rarely meets the technical requirements for appropriate use of significance tests and thus such tests should not be used and of course there are those who hold that significance tests can legitimately be used. There are those who feel that significance testing in social science, even if technical requirements are met, contributes nothing of value to the process of scientific inference and of course there are those that hold that significance testing is a necessary and important aspect of scientific inference.

The Uses of Significance Testing

Professor Henkel has set out the basic logic of significance testing. His background, however, includes much thought and experience connected with the uses and misuses of testing, a topic to which I hope he will return in this discussion. Let me start by offering a few thoughts of my own on this slightly more advanced topic.

As I see it, there are two legitimate technical uses of significance testing in social science, inference to a population parameter and inference to causality. The first is the one that is chiefly covered in Prof. Henkel's introduction. Some would say that the second use, causality, is just a subset of the first one. Be that as it may, I think it's at least convenient to view it separately.

To use testing in the first way, one must have a probability sample, call it a random sample for convenience, of a specific larger population. One calculates a statistic on the sample and uses testing to make an inference to the population. In the overwhelming majority of cases in social science, this inference is about a relationship between two or more variables. Again in the overwhelming majority of cases, the null hypothesis tested is that the relationship in the population is zero. If the result is significant at, say, the 5% level, then we can infer with small chance of error that the relationship in the population is not zero. This use is completely legitimate. It comes up in survey research based on random samples all the time. Note, however, that the test tells us nothing about whether the nonzero relationship in the population is causal. Most survey researchers know that they have to jump through a lot of hoops in order to reach this further conclusion even tentatively.

Social Science Statistics Blog

Last week I discovered (for myself anyway) the Social Science Statistics Blog, which is run by a group of hot-shot graduate students from various departments at Harvard, including Health Policy, Government, Political Science, Sociology, and Statistics.  Most of them appear to have an affiliation with the Institute for Quantitative Social Science.  Their posts have a humor, edginess, and candor that is hard not to like--will they be this interesting when they all get academic jobs?  I hope so.

Here is the introduction to an interesting post by Jens  Hainmueller  (government), entitled The Role of Sample Size and Unobserved Heterogeneity in Causal Inference:

Here is a question for you: Imagine you are asked to conduct an observational study to estimate the effect of wearing a helmet on the risk of death in motorcycle crashes. You have to choose one of two different data-sets for this study: Either a large, rather heterogeneous sample of crashes (these happened on different roads, at different speeds, etc.) or a smaller, more homogeneous sample of crashes (let's say they all occurred on the same road). Your goal is to unearth a trustworthy estimate of the treatment effect that is as close as possible to the `truth', i.e. the effect estimate obtained from an (unethical) experimental study on the same subject. Which sample do you prefer?

See here for the rest of Jens' post.

A Brief Overview of Statistical Significance Testing

My training is in sociology and, though I have an undergraduate background in mathematics and a graduate background in statistics, I do not consider myself a statistician. As a result, my perspective on significance testing may be different from that of the professional statistician and those trained in the physical sciences or business.

To better understand significance tests and their use and misuse, it seems useful to provide a relatively elementary level presentation of both where significance tests fit in the framework of statistics and where statistical inference fits in the framework of scientific inference as well as other contexts in which statistical inference is used. Accomplishing this elementary (oversimplified) presentation involves a consideration of what comprises statistics, some statistical terminology and some criteria that determine if significance tests are properly used, along with a little philosophy of science to set the stage for evaluating the utility of significance testing in, at least, the social sciences. The proper use of significance tests minimally involves discussions of sampling procedures while considerations of the utility of significance testing involves understanding the distinction between statistical and scientific inference.

Blog Forum – Significance Testing

For scholars interested in empirical research, no number is more important than .05. To paraphrase a 1967 article, the difference between .05 and .06 is the difference between right and wrong, creditable and embarrassing, success and failure. See James K. Skipper, Jr., Anthony L. Guenther, and Gilbert Nass, The Sacredness of .05: A Note Concerning the Uses of Statistical Levels of Significance in Social Science, 2 American Sociologist 16-18 (1967). When something is significant at the .05 level, we know it must be important.

Well, maybe this is not quite right. So this week two guests will share their thoughts on significance testing and how best to think about it. First, we have Ramon Henkel of the Department of Sociology at the University of Maryland. Along with Denton E. Morrison, Professor Henkel co-edited The Significance Test Controversy (Aldine 1970), which was just reissued last year. (It includes the previously mentioned Skipper et al. article.) Second, we have Lawrence Mohr of the Department of Political Science at the University of Michigan. Professor Mohr is the author of Understanding Significance Testing (Sage 1990).

A big ELS Blog welcome to them both.

C.J. Robert's "Constitutional Crisis" and the K Street Problem

On page 1 of his 2006 Year-End Report on the Federal Judiciary [hat tip to WSJ Law Blog], Chief Justice John Roberts claims that lagging pay ($165K for a District Court Judge, $175K for a Circuit Judge, $203K for an Associate Justice, $212K for the Chief Justice, see here) has "reached the level of a constitutional crisis that threatens to undermine the strength and independence of the federal judiciary."  (In the 2005 Report, judicial pay was merely a "direct threat to judicial independence.") The Chief Justice's statement is troublesome on its face.  But it is also empirically problematic.

First the text.  Presumably, the "strength" of the judiciary would be undermined by reducing the number of prospective judges.   Most law professors make less than $165,000 per year, as do virtually all civil servants and a large proportion of lawyers in private practice.  This is a pretty deep talent pool.  Why isn't it deep enough?  The Chief Justice needs to explain how another $50K or $100K a year will attract qualitatively better judges and thus avert the constitutional crisis.

What about "independence"?  Are judges who endured an FBI background investigation and congressional hearings really going to curry favor with advocates or litigants in the hope of cashing out with a big law firm or a Fortune 500 company?  Short of bribery, what else could "undermine ... the independence of the federal judiciary" possibly imply?

In the 2006 Report, the Chief Justice argues that the federal bench is on the brink of becoming a mere a stepping stone to lucrative post-government employment.  However, isn't this a problem that affects virtually every government lawyer who works in Washington, D.C.?  Single family homes in Bethesda, or Falls Church, or Arlington, or Georgetown, are prohibitively expensive on a government pay scale; so is private college tuition (if you want to pay cash).  As a result, lawyers at the DOJ, FTC, SEC, EPA, and Capital Hill, etc.,  now routinely parlay their government service into lucrative law firm and K Street lobbying jobs. 

So, if a constitutional crisis is afoot, isn't the cancer even more severe in the executive and legislative branches?  The Chief Justice has lifetime tenure--if this is the true dynamic, he should speak up. 

My empirical analysis, and some interesting graphics, after the jump.

Many Thanks

As my time as guest blogger comes to a close I would like to take some time to thank all of those who  read my sometimes too long posts and I'd especially like to thank those who chimed in with comments. I very much appreciate the opportunity provided to me by the ELS editors to correspond with all of you in this blog forum.  Have a nice weekend! -- Jeff

Josh Wright named FTC Scholar-in-Residence

Joshua Wright, an economist and law professor at George Mason University School of Law (and Truth on the Market blogger), was recently named Scholar-in-Residence at the Federal Trade Commission.   (News release here.)  Josh will be at the FTC full-time next year.  This is a great opportunity for an academic to steep himself in real-world policy issues and better understand the institutional players involved.  In turn, the FTC will benefit from Josh's theoretical and empirical skills  and his numerous connections within the academic community. 

These are valuable synergies, and I hope it is the beginning of a trend.   Congratulations and good luck to Josh.

Explaining Changes in Justices' Ideologies Over Time

A brilliant series of papers have been produced recently on the topic of U.S. Supreme Court justices' ideological change over time. Authors on these papers include Andrew Martin, Kevin Quinn, Lee Epstein, and Jeff Segal, although the author groupings change by paper. Most of the papers can be found on Quinn's website.  To put this very briefly, the authors suggest that - contrary to what prior research claims - Justices' ideological preferences, or ideal points, are not time-invariant, but rather change over time (or at least most of them do).  The authors show that justices' revealed prefrences (i.e. voting) change over time, independent of factors that aren't justice-specific that might affect voting (e.g. changes in the type of cases the Court hears).

In "Assessing Preference Change on the U.S. Supreme Court," Martin and Quinn make a very strong case that justices' ideal points, as revealed in their voting, are not time-invariant. This has important implications for the way we study the Court. However, in the conclusion they posit a very important question that they leave to future research: What explains preference change? We basically, need a measure (exogenous to justice votes) to provide a representation of the justices' sincere ideological beliefs over time. We have time-invariant measures, such as party identification or social background, but coming up with time-varying measures presents a bit of a puzzle.  The authors suggest that we can look toward factors affecting preferences offered by social psychology, stragic considerations, and the macro-political context.

I am working with two UGA graduate students, Brian Levey and Justin Moeller, on the early stages of a project that seeks to answer just this type of question. While we have certainly not "cracked the code," on this puzzle, but we are working toward a strategy for explaining change in ideology over a person's life.  We hope to develop a method of explaining revealed ideological preferences that can be applied to policy makers generally.  Arguments have been made that, for instance, members of congress and presidents change their attitudes over time. Explaining such change is easier said than done, to be sure.

With this goal in mind, I'd like to take a step back and address the basic question - assuming that people do change their sincere ideological preferences over time, what causes such change? Do you know someone whose ideological preferences have changed over time? If so, what do you suspect caused this change? Is it possible that we could apply some of the methods used by Neal Tate (1981) in his seminal article using social background factors (e.g. father's occupation, justice's education, etc.) to explain justices' changing ideological preferences, as revealed in voting?

Of course, there is a rich behavioral literature in political science and other disciplines on how citizens' attitudes are formed, and some literature to suggest why they change. It is possible that justices' attitudes are changed over time by the collective preferences of their social networks. Thus, a change in ones social network might lead to a change in their attitudes or ideological preferences. Other socializing events and phenomena may also prompt change in ideology.

If we consider Supreme Court justices as an example, such events or phenomena (occuring after their confirmation) might be: where they live (change in residence or significant time spent in new surroundings), changes in personal status (marriage, divorce, children, grandchildren, death of a family member), or role on the bench (more senior, elevation to CJ, etc.), among others.  There could also be interactive concerns whereby changes in personal status affect justices differently based upon certain conditioning factors.

Of course, this is all just my early musings on this question, but I would be interested in hearing what other people have to say on this topic.

Postdoctoral Fellowship in ELS

Washington University School of Law Postdoctoral Fellowship in Empirical Legal Scholarship

The Center for Empirical Research in the Law (http://cerl.wustl.edu) at the Washington University School of Law (http://law.wustl.edu/) is offering a one-year postdoctoral fellowship for scholars with Ph.D.s in political science, economics, psychology, sociology, statistics, or other social sciences with interests in empirical legal scholarship.

The Fellow will conduct their own research, participate in an on-going research seminar, and collaborate with law school faculty.  There are no teaching responsibilities.  Candidates must possess a Ph.D. at the time of appointment.  Salary will be competitive and commensurate with experience.  The fellowship will be for a term of one year, with renewal possible.  Applicants should send a curriculum vita, two letters of recommendation, a writing sample, and a statement of planned research to: 

Karma Jenkins (CERL Administrative Coordinator), Campus Box 1120, Washington University School of Law, St. Louis, MO, 63130.  Applications will begin being considered on February 23, 2007.  Please contact Professor Andrew D. Martin (CERL Director) at [email protected] with any substantive questions.

USNWR Gaming and the Failure of Self-Regulation

I recently picked up the latest issue of National Jurist, which is a publication for current and prospective law students.  The cover story this month is "The Rankings Race: How Far Will Law Schools Go?"   

The article provides plenty of examples of how law schools attempt to game their U.S. News input variables, especially with regard to employment.  Michael Sauder, a sociologist at Iowa who has written extensively on rankings, offered instances in which law schools count alumni taxi drivers as "employed" graduates.  David Bernstein (GMU Law) cited a practice in which law schools have rounded up unemployed graduates and employed them for purpose of getting over the nine month USNWR hurdle.

The former President of the Associations of American Law Schools (AALS), William Hines, is quoted that "too many otherwise sensible law schools are engaged in questionable academic practices and may in some cases be borderline dishonest."  Two years ago, a similar damning story appeared in the New York Times.  See Alex Wellen, The $8.76 Million Maneuver.   

I don't think the legal academy appreciates the implications of all this well-documented gaming.  Not only are we cutting corners writ large in order to get ahead--and hence have zero credibility to emphasize the importance of honest, integrity, and ethical behavior to our students--but we are providing overwhelming evidence that we are incapable of effective self-regulation.  Right now, gaming pays.  Further, if you are a dean, and you refuse to aggressively manage your USNWR ranking, your job is in serious jeopardy.  But calling U.S. News a collective action problem is a cop-out.  Where is the ABA, our accrediting agency, to police this behavior?   

The table below, from the Measuring Outcomes paper by Andy Morriss and I, documents changes in graduate placement since U.S. News moved to the current employed at graduation/9 months methodology.  If these numbers are to be believed--and we would advise against it--these are bullish times for newly minted lawyers.  And the rising tide has raised all tiers.

Some of these remarkable gains may be the result of competition among schools to more effectively place, or at least track, their graduates.  If that is the primary explanation, then U.S. News has actually been a force for good, at least in terms of employment.  But a more likely explanation is that any employment--law firm associate, judicial clerk, public interest lawyer, taxi driver, waiter, insurance salesman, temporary law school RA, whatever--counts for the purposes of U.S. News. 

The biggest red flag is that while total employment at 9 months for all law school graduates surged from 83.9% to 91.6% between 1997 and 2006, first-time bar passage for this same cohort dropped from 83.0% to 78.6%, presumably due to the trend toward higher bar cut scores.  See pp. 13-14.

Note that the results from a February bar will rarely be available within 9 months of graduation. So, as we note in the paper,

it is likely that much of the increased post-graduation employment is in jobs that do not require a valid license to practice law and hence are unlikely to pay a premium for three years of legal education. At a minimum, this practice has the potential to seriously mislead prospective law students attempting to evaluate the relative costs and benefits of a legal education. It is also hard to believe that this type of reporting should be considered sufficient to satisfy ABA Accreditation Standard 509, which requires a law school to provide "basic consumer information" in a "fair and accurate manner reflective of actual practice."

As a self-regulating profession, it is time to get our own house in order.

Family Leave in Law Schools

Laura Kessler (Utah) has written Paid Family Leave in American Law Schools: Findings and Open Questions.   The Abstract:

"There exists a substantial literature on the status of women in legal education, including studies on women students' experiences in law schools and gender bias on law school faculties. However, no recent study examines the family leave policies and practices in American law schools. This pilot study of thirty-three law schools represents an initial step toward filling that gap. Its findings are threefold. First, almost three-quarters of the law schools surveyed provide some form of wage replacement during a family leave that is more generous than required by federal law. Second, among the law schools surveyed, there is a positive relationship between teaching at top-tier and private law schools and receiving a paid family leave. Third, although many of the surveyed law schools provide paid family leave, most leaves were granted on a case-by-case basis. This study stands to benefit a number of constituencies within the legal academy, including individual faculty members, faculties developing leave policies, appointments committees, and job candidates. While preliminary in nature, it raises a number of open empirical and theoretical questions for future research."

ELS and the question of Refereed vs. Student Edited Law Review

I have seen this question discussed in general on a number of blogs, but I am particularly interested in what empirical legal scholars thought about the benefits and detriments of both types of publishing outlets. To some degree, your views may turn on what your institution rewards (law reviews or peer reviewed journals), but there is also the matter of what journals you find publishing in most rewarding personally and intellectually. While there is increasing cross-over in both publishing and reading in the two camps, I think that you do make an important decision about the potential audience for your work when you choose one over the other.

This being said, there are a number of what are reasonbly considered law review outlets that are peer reviewed or refereed.  In reviewing John Doyle's excellent law review rankings and submissions webpage, I count over 200 law outlets that are refereed, including the Supreme Court Review, Journal of Legal Studies, American Business Law Journal, Law & Society Review, and the Journal of Empirical Studies, among others.  Of course, there are a multitude of other referreed outlets in discplines such as political science, economics, criminology, public administration, and sociology, to name just a few.

I would be interested in hearing what others think about the relative advantages and disadvantages of publishing in student edited law reviews vs. peer reviewed journals. Here are what I believe to be some the advantages of each:

Student edited law reviews:

1) Simultaneous submission. Perhaps nothing is more annoying than having to wait six months or more for the slow wheels of the peer review to render a verdict - and then on to the next journal for another slow, nail-biting wait. The sequential process of peer review is especially hard on untenured people, for whom timing is very important.
2) A larger, and more diverse audience. This might be arguable, but I believe that most will at least agree that law review exposure usually brings you a new audience for your ideas, relative to the rather limited audience of some social science disciplines.
3) Greater academic freedom. There, I said it. For whatever reason, it seems that the peer review process somehow tends to limit the type of questions that people think are "interesting," and therefore worthy of publication.  It is my impression that law reviews are more open to different approaches and new research topics. I'm quite sure that many will disagree with me on this point.

Peer Reviewed Journals

1) A more rigorous review process. There, I said it. I think that good things can come from having peers review your work and I believe that many of my publications are much better because of the peer review process.  Further, with regard to ELS, some may question the capability of student editors to assess high level empirical methods. That being said, it is my understanding that informal networks of peer review in legal academia go a long way toward addressing this concern.
2) A blind review process. I've seen some discussion on the lack of blind review with regard to law review publishing and I think that the "academic celebrity factor" in law review publication decision making is a bit hard to deny. Balkin and Levinson's witty and insightful article "How To Win Cites and Influence People" is especially revealing on the law review publication process and gaining citation. To be sure, the blindness of the review process in refereed journals is perhaps open to question and the development of internet search engines hasn't helped this concern.
3) It keeps you humble. I really couldn't think of a good third advantage, but I wanted to mantain symmetry of the categories. However, peer review certainly does keep you grounded - nothing like a blistering (and insightful) review to remind you that you don't know everything.

I'm sure that you all will have plenty to say about this. If I have offended people from both sides equally ...  well then, my job is done. So, in the words of Saturday Night Live's Mike Myers, playing his mother-in-law, here's the topic ... "Refereed vs. Student Edited" .... talk amongst yourselves.

Second Annual Conference on Empirical Legal Studies

Click here for the announcement for the Second Annual Conference on Empirical Legal Studies, which will be held at New York University on November 9-10, 2007. The conference's official website is at: http://www.law.nyu.edu/cels/.  The paper submission deadline is July 1, 2007.

Cross-Monitoring and Corporate Governance

I just finished reading a really terrific empirical study by Joanna Shepherd (Emory Law), Fred Tung (Emory Law), and AlbertYoon (Northwestern Law), entitled Cross-Monitoring and Corporate Governance.  In a nutshell, the authors show, after controlling for a wide variety of relevant control variables (most importantly corporate governance), that companies that rely on bank loans, as opposed to equity or public debt, post higher market returns.  Presumably, these results flow from the heightened monitoring and positive market signaling supplied by loan agreements.  Their multiple specifications do a nice job of ruling out alternative hypotheses.

This is quite an important paper for thinking about manager and director agency costs, especially in a world where portfolio diversification has reduced the incentives for shareholders to pay attention to annual elections for directors.  (It also is a excellent example of methodology for large panel data.)  Here is the article abstract:

We take the view that corporate governance must involve more than corporate law. Despite corporate scholars' nearly exclusive focus on corporate law mechanisms for controlling managerial agency costs, shareholders are not the only constituency concerned with such costs. Given the thick web of firms' contractual commitments, it should not be a surprise that other financial claimants may also attempt to control agency costs in their contracts with the firm. We hypothesize that this cross-monitoring by other claimants has value for shareholders. 

We examine bank loans for empirical evidence of the value of cross-monitoring. Our approach builds on prior empirical work on the value of good corporate governance, to which we add data on the presence of bank loans and their interactions with free cash flow, governance indices, and individual corporate governance provisions. We find strong evidence that bank monitoring adds value. In effect, bank monitoring can counteract somewhat the value-decreasing effects of managerial entrenchment. Bank monitoring may substitute for good corporate governance.

Summer Institute - EITM VI

UCLA will be hosting the sixth summer institute on the Empirical Implications of Theoretical Models (EITM) from June 24 to July 21, 2007:

The Summer Institute on EITM seeks to train a new generation of scholars who can better link theory and empirical work. It is a highly interactive training program for advanced graduate students and junior faculty led by political scientists from across the discipline who employ EITM in their research. Summer institutes general[ly] accept 25 advanced graduate students and junior faculty through a competitive selection process. In most cases, tuition, room and board, travel and living expenses are covered for participants through a grant from the National Science Foundation.

Graduate students who will benefit most from the program should be committed to using both theoretical models and empirical data in their dissertations. They should have some training in both formal methodology and quantitative analysis -- and advanced training in at least one of these areas. We will also accept junior faculty looking to improve their defended dissertation in a direction that incorporates EITM, or junior professors that are embarking on a "second EITM-like project".

There is more information here.

Trial courts, judicial politics, and summary judgment ...

First, I'd like to thank the editors of ELS for asking me to guest blog. I think that blogs can provide valuable points of intellectual interaction and can also be a lot of fun.

A while back, Washington University law professor Margo Schlanger blogged on ELS about trial courts and trial court judicial politics, and I was hoping to revisit this topic. Her Civil Rights Litigation Project goes a long way toward facilitating research on trial courts. Trial courts and access to justice have long been research interests of mine. Between other research projects and teaching, I have been doing some reading on the use of summary judgment by trial court judges and especially empirical examinations of summary judgment decisions.

I was especially intrigued by Harvard law professor Arthur Miller's article in the NYU Law Review - "The Pretrial Rush to Judgment: Are the 'Litigation Explosion,' 'Liability Crises,' and Efficiency Cliches Eroding Our Day in Court and Jury Trial Commitments?" (2003). It brought up some things that I had thought about during my attorney practice years and had continued to think about during my years of looking at courts from a social science perspective. I won’t go into a full recap of his excellent 150+ page article, but he is basically concerned with the shift in responsibility between jurors and judges in making determinations of fact. He (along with other scholars) even suggests that such a shift, via expanded summary judgment, might undermine the Constitutional right to a jury trial.  Certainly, expanded use of summary judgment has important implications for citizens' access to justice.

Guest Blogger: Jeff Yates

This week's guest blogger is Jeff Yates, Associate Professor of Political Science at the University of Georgia.  Jeff received his J.D. from the University of Tennessee and his Ph.D from Florida State University.  His research focuses on the study of judicial process, criminal justice and tort litigation.  His recent work is available here.

How to Teach ELS in Law Schools

A number of ELS gurus have blogged about empirical courses offered by law schools. I'd like to raise some issues about teaching ELS in law schools.

1. What are we trying to do? Is the goal to teach statistics to students with no background so that they understand empirical evidence? Or are we trying to go further than that and enable students to produce statistical evidence and/or understand academic ELS? Ideally, the answer would be both, but its hard to do this with one course, and right now one course is the high end for the number of ELS courses offered at any law school.  With time, law schools may begin offering ELS sequences with more than one course, which would help solve the problem, but right now any ELS course has to decide what its goal is.

2. How do we get the right people to take the right course? If law schools are only offering one course on ELS, then it seems to make sense for the course to be introductory. One concern with this is that the course may be disproportionately taken by people with prior backgrounds in the subject-- the course will be reteaching stuff that most of the students had heard already, rather than introducing new material to those who would most benefit. Assuming law schools will be reluctant to make statistics mandatory, I 'm not sure what the solution should be.

3. How do we teach ELS? Should it be taught with an ordinary statistics book, or is there something to be said about greater integration of the law into a course. Its easy to say that law should be integrated into the course, and that is certainly the preferred option, but how do we go about doing it? Is there a textbook that does this? When its all said and done, it might be easiest to teach the skills directly. 

Any and all comments are greatly encouraged.

Law and Public Health Faculty Opening

Michelle Mello asked that I post the following:

FACULTY JOB OPENING: The Harvard School of Public Health is seeking candidates for a tenure-ladder position as assistant or associate professor of law and public health.  The faculty member will work on collaborative health law research projects with a focus on empirical research.  Information is available at http://www.hsph.harvard.edu/searches/hpm1.html.

Deadline for Law Firms Working Group

Back in November, I posted an announcement for the Law Firms Working Group, which is a joint initiative of the American Bar Foundation and Indiana Law.  Pursuant to a special licensing agreement with American Lawyer Media (ALM), the Working Group is soliciting proposals for research projects that will utilize a large cache of data on law firms, including information on law firm profitability, revenues, billing rates, firm size, structure, office locations, various dimensions of associate satisfaction, lawyer demographics,  technology migration, recruitment of new associates, lateral partner mobility, corporate clients, big deals and big lawsuits, and much more.  Most of the variables are available for multiple years.   

ALM is remarkably good at collecting data, but academic researchers--i.e., ELS readers--have a comparative advantage when it comes to analyzing it.   This is the core insight that produced the special licensing arrangement.  In my capacity as the Working Group director, I have received numerous inquires and several proposals.  If you are interested in submitting a proposal, the deadline is February 16.  Note that we are limited to twelve active projects.  See this announcement for details.

ELS and the ESA

I always enjoy it when environmental law and empirical legal studies find common ground, so check out the intriguing Abstract of Learning from Endangered Species Recovery Programs: A Case Study Using U.S. Endangered Species Act Recovery Scores:

"Threatened and endangered species recovery programs consume increasing resources. Even so, there is increased concern about actual and projected biodiversity losses and in the success of recovery programs in reversing these trends. In this paper, we use a panel data set and ordered probit econometric methods to statistically examine the determinants of the 1990-2002 biennial U.S. Fish and Wildlife Service (FWS) recovery scores for up to 225 vertebrate species listed as threatened or endangered under the Endangered Species Act. We find that species-specific spending is a significant determinant of species' recovery scores and that increased spending reduces the probability that FWS will classify a species as extinct or declining. The evidence does not support the hypothesis that increased spending increases the probability that a species is stable or improving. Other FWS' actions have significant and substantive influences on improved recovery scores. These include progress on or completion of a recovery plan and achievement of stated recovery objectives. We find evidence that species achieve better recovery scores if FWS considers them to have high recovery potential and that species whose recovery is judged by FWS to be in conflict with economic activity are more likely to be classified as extinct. Our evidence does not support the conclusion that critical habitat designation promotes species' recoveries or prevents species' declines. We also report a new finding that recovery success varies across FWS administrative regions."

State Corporate Tax Evasion

In 2005, Citizens for Tax Justice  released a report  on effective state corporate income tax rates entitled "Corporate Tax Avoidance in the States Even Worse Than Federal."  The most prominent conclusion of the study is that:

A new analysis of the state corporate income taxes paid by 252 of America’s largest and most profitable corporations finds that by 2003, these companies on average failed to include two-thirds of their actual U.S. pretax profits on their state tax returns.

By 2003, these 252 companies had slashed their state income tax payments to an average of only 2.3 percent of their U.S. profits. Since the average statutory state corporate tax rate is about 6.8 percent (weighted by gross state product), that means that in 2003, two-thirds of their profits escaped corporate tax entirely.

This is an important study of corporate income tax rates-- a subject we know little about-- and I don't doubt the finding that two-thirds of corporate profits avoided state tax. I am less certain, however, that this is mostly because companies set out to avoid state corporate income taxes.

For companies that operate in multiple states, corporate income must be apportioned to each state. States do this through a formulary system. In 1957, the formulary was based on an equal weighting of sales, property, and payrolls. As I understand it (please correct me if I'm wrong), this means that if a company had 10% of its sales, 33% of its property, and 60% of its payroll in Alabama ,and 90% of its sales, 67% of its property, and 40% of its payroll in CA, then 36.7%=(.333*.1)+(.333*.4)+(.333*.6) of that corporation's income would be apportioned to Alabama and 63.3%=(333*.9)+(.333*.6)+(.333*.4) of its income to California. Many states have changed the formula, however to overweight sales. In a state like Alabama, this would reduce the corporation's state tax burden. For example, if Alabama placed a two thirds weight on sales, then only 23.3%=(.667*.1)+(.167*.4)+(.167*.6) of that corporation's income would be taxed in Alabama.

If every state followed the same formulary (and doubled the weight on sales), then on a national level there would be no decrease in tax. The company would pay less of its tax in Alabama, and more tax (76.7%) in California where sales are high. Every state doesn't follow the same formulary, however. For some background, see pages 14 and 15 of this article. Indeed, I think that states with high sales typically do not place added weight on sales. If California sticks with the original formula while Alabama deviates, then only 86.6%=63.3%+23.3% of corporate income will be attributed to some state.

Thus, the effective state corporate tax rate will be lower than the average state corporate tax rate, not because comapanies are deliberately avoiding corporate income taxes, but rather because the formulary inconsistencies allow some corporate income to be effectively untaxed. I don't know about the size of this effect, and it may be small, but if the formularies are highly skewed, it is easy to imagine that a lot of corporate income doesn't get taxed in any state without any deliberate avoidance by corporations. Further empirical work on this issue could easily get at the size of this effect.

State Tort Reform Database -- User-Friendly Updates

Last summer I posted about Ronen Avraham's (Northwestern) database containing state tort law reforms for all fifty states (and DC) during the recent decades.  Ronen recently mentioned updates to that database that make it even more user-friendly, especially as it relates to coding.  Those interested in the updated database can access it (and the related paper) here.

Financial Distress and Investment

One of my favorite articles  about the costs of bankruptcy is by David Cutler and Larry Summers (yes, that Larry Summers). Here is the abstract of the article:

Since 1984, Texaco and Pennzoil have been engaged in a legal battle over Texaco's usurpation of Pennzoil in the takeover of the Getty Oil Company. The stakes are huge: the jury award that has been upheld through several appeals calls for Texaco to pay Pennzoil more than $10 billion. The Texaco-Pennzoil case presents a unique natural experiment for studying debt burdens and bargaining costs. Essentially continuous market assessments of the prospects of both parties in a high stakes bargaining game are rarely as observable as they are in the case of publicly traded companies like Texaco and Pennzoil. Further, unlike in the Texaco case, financial distress is usually brought on by events impinging directly on a firm's operations, thus making the costs of distress difficult to measure.

This paper uses data on the abnormal returns earned by the shareholders of Texaco and Pennzoil to examine whether resources were "lost" in the course of the litigation. We find that the leakage involved in the forced transfer is enormous: each dollar of value lost by Texaco's shareholders has been matched by only about 30 cents gain to the owners of Pennzoil. Our estimates suggest that the Texaco-Pennzoil conflict has reduced the combined equity value of the two companies by about $2 billion. Further losses have been suffered by Texaco's bondholders, though these may be offset by the tax collections that would result if Texaco made a large payment to Pennzoil.
After documenting the large joint losses that Texaco and Pennzoil have suffered, we seek to identify their causes. Clearly one explanation is the fees that both companies will pay to the many lawyers, investment bankers, and advisors that have been retained. Even making generous allowance for these costs, however, we are unable to account for a large fraction of the loss in combined value. It appears that there have been additional costs to Texaco's shareholders from disruptions in Texaco's operations, difficulties in obtaining credit, incentive problems created by fears that Texaco would cease operations, and distraction of top management. 

(Note-- I'm biased in favor of this article--I've used the methodology first employed by Cutler and Summers to examine the efficiency of specific performance ). 

The argument that "difficulties in obtaining credit, incentive problems created by fears that Texaco would cease operations, and distraction of top management." hurt value is reasonable, but the paper doesn't provide much evidence of this. This has always bothered me. Today, I'll search for some evidence that fits in with a long finance literature on the impacts of the impacts of financing contstraints on investment.  One way the difficulties caused by the verdict might have killed value is by limiting investment. If Texaco's managers were distracted and Texaco's access to outside credit was limited, then we'd expect its investment to go down. If Texaco is passing up good investment opportunities because of the litigation, then this is a real cost of the litigation that might explain the drop in combined value of Texaco and Pennzoil.

So did investment at Texaco go down?

To answer this, I took a look at capital expenditure rates at Texaco and other oil companies (in the same NAICS code) between 1980 and 1994. I'll use a basic differences-in-differences  framework:

Inv(year i,company t)=a+b*Texaco_dummy+c*post_trial_dummy +d*Texaco_dummy*post_trial_dummy+error term

The key coefficient is d, whether or not something special happened to Texaco's investment after the  post trial verdict. Note that the investment variable is scaled by the size of the company to allow for comparison.

The results are inconclusive-- not surprising when there is only one variable receiving the treatment.  Texaco's investment indeed went down after 1984-1985. In 1980-1983, Texaco's capital expenditures equalled around 11% of gross PPE. In 1986-88 (the initial verdict was in late 1985), Texaco's capital expenditures went down to around 5.5% of hard assets. It sound like a big decline, confirming the Cutler-Summers speculation. The problem is that the capital expenditures of other oil companies went down even more, from around 15% of gross PPE in 80-83, to around 8% of gross PPE in 86-88.

The estimate of d is actually positive (around .01)-- Texaco's investment went down less than other company's investment after the verdict-- but the coefficient is nowhere near significant.

If the verdict had a bad effect on Texaco's capital expenditures, we can't find that effect in the midst of the heavy decline in investment for all oil companies during the period. Cutler and Summers' speculation about the cause of the value decline for Texaco remains speculation.

 

More Perspectives on IRBs

Those interested in a slightly more sanguine perspective on the role of IRBs (especially in the bio-medical setting) will find Jerry Menikoff's (Kansas, Medicine) recent paper helpful (forthcoming in a Northwestern L. Rev. symposium issue).  An excerpted abstract follows:

"... Admittedly, the current system is imperfect and a variety of improvements can and should be made to eliminate those review requirements that do little to protect subjects. But even with these problems, getting IRB approval of social and behavioral research studies should in the great majority of cases be a relatively non-burdensome task that is a minimal hindrance to the conduct of the research. That burden should rarely if ever rise to the level of triggering constitutional protections against censorship."

Pensions and Risk Aversion: The Influence of Race, Ethnicity, and Class on Investor Behavior

Dorothy Brown (Washington & Lee) has written Pensions and Risk Aversion: The Influence of Race, Ethnicity,  and Class on Investor Behavior.  The Abstract:

Defined Contribution plans have greatly expanded over the last two decades. Defined Contribution plans place the investment risk on employees. Employee investment decision making should be examined to determine whether those decisions are influenced by race, ethnicity and/or class.

Empirical data show that investor behavior is greatly influenced by race, ethnicity and/or class. Blacks and Hispanics are far less likely to invest in the stock market than whites. Low-income whites are far more likely to invest in the stock market than blacks or Hispanics regardless of income. As a result, retirement account balances are the greatest for many white households and the least for black, Hispanic, and certain white households. This article explores those issues and suggests solutions that will allow employees to overcome their built-in biases and make wiser investment choices.

Regression Discontinuity Design

I'm a big fan of regression discontinuity studies.  In a regression discontinuity study, there is typically a discontinuous determination of who receives a treatment. For example, a cholesterol medication may be given to all individuals with bad cholesterol levels over x, but to no individuals with cholesterol levels below x.

In a regression discontinuity design, we exploit the discontinuity to determine the treatment effect. The idea is that some individuals randomly fall just above the treatment cutoff (in my example, they have a cholesterol level of x+epsilon) while others fall just below (have a cholesterol level of x-e). These individuals are pretty similar-- their cholesterol levels are almost identical, yet they get treated differently. Some get the treatment, but others don't.  If, after receiving the treatment, a group of people with original cholesterol levels just above x have a much lower cholesterol level than a group with original cholesterol levels just below x (who do not get the medication), then we can attribute that decrease in cholesterol level to the medication.

It is, of course, true that the people who receive the medication have a higher average cholesterol level than those who do not, but the size of the effect is small and can be controlled for.

When regression discontinuity assumptions are valid, then the design is a great way to obtain valid inferences of causal effects in a non-experimental setting.   For a really cool application of regression discontinuity to estimate the impact of class size on test scores, see Angrist and Lavy (1999).

The validity of the identification from regression discontinuity designs comes from the assumption (among others) that people with cholesterol levels just below x are very similar to those with a cholesterol level just above x. This assumption will be satisfied when people are unaware of the cutoff, or if the cutoff is not important enough to induce any change in behavior. When making or not making the cutoff is an important choice variable, however, then the validity of the inferences may be flawed.

For example, suppose that people know x and that this medication is highly anticipated-- some people really want to be able to take the medication. Suppose also that you can regularly check your cholesterol and affect your level by modifying behavior. If this is the case, then people with cholesterol levels just below x might be very different from people with cholesterol levels just above x. The ones with cholesterol levels just above x might be people who really wanted the medication and made sure to have just enough cholesterol, but not more than that. The ones with cholesterol levels just below x apparently didn't care about getting the medication or were "bad" at controlling their cholesterol levels. Under these assumptions, the groups are very different in their attitudes towards medication and/or their motivation levels, in spite of the fact that their cholesterol levels are very similar.

When we apply the regression discontinuity design in this case, then we do not get the valid inferences of the impact of the medication that we'd like. The difference in post-treatment cholesterol levels between those with original cholesterol levels just above vs. just below x may be due to the medication, but they also may be due to other factors, such as differences in motivation or difference in belief in the power of medication.

At the recent AFA meetings in Chicago, there was a very interesting paper by Chava and Roberts that may be subject to this critique. The paper tries to estimate the impact of violation of debt covenants on the amount of corporate investment. The paper applies a regression discontinuity design. Many covenants define minimum net worths. The paper estimates the effect of violating the covenant by comparing the investment decisions of companies that barely violated net worth covenants with the decisions of companies that barely avoided violations. The paper assumes that these companies are similar except for the fact that some violated the covenants and others didn't.

I'm not so confident that this assumption is valid. I presume that most companies want to avoid covenant violations. Once you've avoided a violation, however,  then you care much less about your actual net worth and don't spend much time worrying about it. This means that companies that barely violate the covenants may be systematically different than those that don't violate the covenants. The companies that violate may less careful, more honest, or be in a much more delicate financial position than those that are just above the minimum new worth. The effect found by Chava and Roberts may be the impact of covenant violations, or it may be due to the fact that the two groups they are comparing are systematically different. 

I don't want to sound too negative, however. Regression discontinuity is a great research design, and I'd love to see more of it in empirical legal studies.

Guest Blogger: Yair Listokin

This week's guest blogger is Yair Listokin, an Associate Professor of Law at Yale Law School. Professor Listokin received his J.D. from Yale Law School in 2005, his A.B. magna cum laude in Economics from Harvard in 1998, and his Ph.D. in Economics from Princeton in 2002. At Yale Law School, Professor Listokin was three times named the John M. Olin Prize Winner for the best student paper in law and economics, and he was a John M. Olin Fellow in Law and Economics. In 2005-2006, he clerked for The Honorable Richard A. Posner at the U.S. Court of Appeals, 7th Circuit. His primary research and teaching interests are business organizations, bankruptcy, contracts, corporate finance, and empirical legal studies. His additional interests are in federal income taxation and secured transactions.

More on Judicial Independence

For those in the D.C. area:

JUDICIAL INDEPENDENCE: DRAWING LESSONS FROM HISTORY

A panel discussion including:

Honorable Howard H. Baker, former U.S. Senator and U.S. Ambassador to Japan Honorable Paul D. Clement, Solicitor General of the United States Professor Vicki C. Jackson, Georgetown Law Center Honorable Ann C. Williams, U.S. Court of Appeals for the 7th Circuit
Moderator: Russell Wheeler, President, Governance Institute

Thursday, January 25, 2007.  3-4:30pm
Thurgood Marshall Federal Judiciary Building, Washington, D.C.

RSVP: [email protected] or 202-479-3411

Lott v. Levitt: A Partial, Implied Vindication for Lott?

The court dismissed Count I because the sentence on page 134 of Freakonomics is capable of an innocent construction. The relevant construction is based on the reasonable reader, and the court determined that the reasonable reader is based on the general population, not the academic population (though I don’t think it matters). The court did NOT say that it would have come to a different conclusion had it applied a standard based on the academic population, but one might draw this implication from the opinion.

The following excerpt comes from Robert’s Rationale, a non-academic blog I just discovered through Google (and one I don’t know much of anything about beyond this post):

The judge ruled against the first allegation. The word “replicate” does have a specific meaning in science: repeating an experiment precisely. However, the book was in large part marketed to the general public, which would take the sentence to mean that other scientists, using different methods and statistics, have come to different conclusions.

In defamation law, there’s a practice of “innocent reading,” that is, if a statement can be taken two ways and one of them is not libelous, the non-libelous one should prevail. What’s interesting here is that there are two audiences; one could read it two ways, but the other could only read it in the libelous way. The judge ruled that the general public’s reading was the one that counted, even though many academics also read the book (it is, after all, an application of econometric methods).

Lott himself makes a related point on his own blog about the popularity of Freakonomics with academics: “I think that the market for the book was also aimed at academics. The book is apparently marketed to a large number of economics classes and is read by academics.”

So, will the popular view of this case be that Levitt defamed Lott to his smaller, academic audience, but that he was not liable because he wrote the book primarily for a larger, general audience?       

Lott v. Levitt - Count I Dismissed

Yesterday, Judge Ruben Castillo dismissed Count I of Lott’s defamation lawsuit against Levitt but not Count II. Count I is based on the sentence on page 134 of Freakonomics; Count II is based on Levitt’s e-mail to economist John McCall.

In the discussion of Count I, Judge Castillo determined the sentence is not defamatory because it is capable of an “innocent construction.” He said:

Lott claims that the following sentence in Freakonomics is defamatory per se: “When other scholars have tried to replicate [Lott’s] results, they found that right-to-carry laws simply don’t bring down crime.” Lott argues that the only reasonable meaning of the last sentence is that he falsified his results because “the term ‘replicate’ has an objective and factual meaning in the world of academic research and scholarship.” Specifically, Lott claims that the “clear and unambiguous meaning” of “replicate” is that “other scholars have analyzed the identical data that Lott analyzed and analyzed it the way Lott did in order to determine whether they can reach the same result.” By claiming that other scholars have tried to “replicate” his research and results, but come to a different conclusion than Lott, Lott claims that the sentence in Freakonomics alleges that “Lott falsified his results.”

The applicable standard, however, is not that of the “world of academic research and scholarship” that Lott describes. Rather, the critical question is how a “reasonable reader” would interpret the phrase. The reasonable reader in this case is the general population, who helped make Freakonomics an “extraordinary successful,” “best-selling book.” In everyday language, replicating “results” does not necessarily mean analyzing identical data in identical ways, and thus it is reasonable to read the sentence at issue as not accusing Lott of falsifying his results. In fact, it is more reasonable to read the sentence as stating that other scholars testing the same hypothesis have done separate research, with possibly different data and statistical analyses, and come to different conclusions, thus disproving Lott’s theory; or simply, that other scholars attempted to arrive at the same conclusions as Lott had, but were unable to do so.

Lott v. Levitt, No. 09 C 2007, slip. op. at 6-7 (N.D. Ill. filed Jan. 11, 2007) (internal citations omitted).

The court referred to Levitt’s e-mail to McCall as “unfortunate and ill-considered” and refused to dismiss Count II.

John Lott has a copy of the decision on his website. Lott’s response: “I think that the market for the book was also aimed at academics. The book is apparently marketed to a large number of economics classes and is read by academics.”

H/T to Tim Lambert

"A Conversation on Judicial Independence"

Our Milwaukee-area, Chicago-area, and Wisconsin readers may be interested in the following:

The Milwaukee Lawyer Chapter of the American Constitution Society invites law students to "A Conversation on Judicial Independence," featuring the Honorable Patience Drake Roggensack, Wisconsin Supreme Court, on Jan. 25, 12:00 - 1:30 p.m., at the Milwaukee Bar Association, 424 East Wells St. There is no cost to attend this event. Lunch will be provided. Please RSVP by completing form at http://www.acslaw.org/chapters/lawyer/milwaukee/rsvp.

Justice Patience Drake Roggensack of the Wisconsin Supreme Court will discuss the philosophy underlying judicial independence, its importance and effect in the three tiers of the Wisconsin court system, its relationship to the judiciary's interaction with the other branches of government, and how it relates to the development of common law doctrines, statutory interpretation and constitutional interpretation.

2007 Statistical Abstract Available

The newly released Statistical Abstract provides a portrait of the American people drawn from the 1,376 tables provided.  According to the U.S. Census Bureau website, "The Statistical Abstract of the United States, published since 1878, is the authoritative and comprehensive summary of statistics on the social, political, and economic organization of the United States."

2007 Law & Society (Berlin): Proposal Deadline

Just a quick reminder that proposals for the 2007 Law & Society Association Annual Meeting in Berlin (July 25-28) are due tomorrow (Fri. Jan. 12).  L&S meetings remain a traditional and important forum for empirical legal scholarship.  As well, I've always felt that L&S meetings are especially helpful for younger scholars and do a good job of attracting research from an array of disciplines.  More info here.

What Exactly Do We Know from the Empirical Study of State-Level APAs...

Jim Rossi has written Politics, Institutions and Administrative Procedure: What  Exactly Do We Know from the Empirical Study of State-Level APAs,  and What More Can We Learn?.  The Abstract:

Empirical research to date on state-level administrative procedure acts (SLAPAs) is inconclusive regarding when states will adopt and amend SLAPAs. However, there is reason for optimism that more comprehensive empirical study could shed light on the McNollgast hypothesis and its variations - allowing researchers to better understand the relationship between commitments, politics, and administrative procedure. This will require the assembly of much more exhaustive and comprehensive data than has previously been collected. Once data is assembled, empirical study could shed light on: a) the extent to which administrative procedures are neutral or whether they are likely to have a partisan bias; b) the effects of divided government on the adoption of administrative procedures; c) the influences of the length (or durability) of procedural commitments in SLAPAs; and d) the extent to which model statutes are effective in minimizing partisan influence in the adoption and amendment of SLAPAs. This Essay addresses how the collection and analysis of such data might proceed, and how such analysis could open up possibilities for important future research, including research on the effects of administrative procedure reforms in states. The Essay also argues that there are benefits to having administrative lawyers participate with social scientists in the empirical enterprise of examining SLAPAs.

Cutting Your Empirical Teeth with LSSSE Data

During the AALS annual meeting, I was fortunate to be panelist on the program organized by the Committee for Research,  entitled " Everything We Wish We Knew About The World of Legal Education: An Agenda for Research."   One of the items discussed by the panel was the research value of data from the Law School Survey of Student Engagement (LSSSE). 

LSSSE is the law school version of the National Survey of Student Engagement (NSSE), which is utilized by over 50 percent of all undergraduate institutions.  In short, LSSSE and NSSE are rapidly becoming the leading method quality control within a competitive educational marketplace.

For the past five years, LSSSE has surveyed law students at 110+ ABA-approved law schools on a wide variety of topics, including student-faculty relations, type and volume of feedback from professors, time devoted to studying, student evaluations of learning gains, allocation of time between school, work, family, and co-curricular activities, etc.  The 2006 LSSSE survey instrument can be found online here  (note:  each question equals one or more variable in the dataset).

As I said during the panel--and reiterate here for ELS readers--LSSSE data presents a great opportunity for legal academics to cut their teeth with empirical methods.   Assuming your law school is one of the 110+ that have participated, the following is available to you:

• A file in SSPS format (easily converted to Stata) with a detailed codebook;
• The ability to match respondents with admissions data (LSAT, UGPA, major, age) , law school record (grades, courses taken), and outcomes (bar passage, employment information);
• A research context--i.e., student learning environment--that does not require much of a learning curve for law professors.  In other words, interpreting the statistics will be a little more intuitive and a little less intimidating;
• A rich dataset that could answer lots of important questions on what works and what doesn't in the legal academy--i.e., your study will get read;

(Note that if you want to publish your results, you will need to get IRB approval from your home institution before you commence your research.)

Fortunately, here at Indiana Law, we have participated in LSSSE for five consecutive years and hence have a terrific longitudinal dataset.  Ken Dau-Schmidt, Jeff Stake, and I recently drafted a research design for LSSSE data, which I presented at a separate AALS panel ("Learning, Educating and the Law School Years").    Below the fold are three slides that (a) identify LSSSE claims, (b) summarize our basic theoretical assumptions, and (c) state our two primary research questions.    

Official Blog of the AALS Section on Law and Social Sciences

A note:

After today’s AALS Section on Law and Social Sciences panel, I, Jason Czarnezki, was voted Secretary of the Section, and, most importantly, the Empirical Legal Studies Blog was pronounced the Official Blog of the American Association of Law Schools Section on Law and Social Sciences.

If there is any Section business or announcements that anyone would like conveyed to the ELS community, please email me at [email protected], and I will post this information on the blog.

"Designing Empirical Legal Inquiry”

At the AALS Annual Meeting today, the Section on Law and Social Sciences organized a panel entitled, “Designing Empirical Legal Inquiry.”  The four panelists were Andrew Martin (Wash U), Lee Epstein (Northwestern), Tracey George (Vanderbilt), and Ahmed Taha (Wake Forest). The panel was designed for each of the participants to discuss different empirical methods to address the same question—How do judges make decisions?

Epstein kicked off the panel by discussing the Dark Side of Archival Research.  While she normally reports about the rewards of archival treasure hunts, she wanted to make clear that while worthwhile, archival work requires money, includes uncomfortable workspace and lots of photocopies, and mandates the creation of detailed coding rules.

STATA Graphics

Query:  What are the best reference texts for basic STATA graphics?   Comments are open.

Rankings Debate at AALS

A rush of deadlines have cramped my blogging over the last few weeks, including one for the AALS Workshop on the Ratings forum.  My co-author, Andy Morriss, was one of the panelists.

The responses to the various rankings panels and breakout sessions elicited lots of frustration and  high horse moralizing.   As someone who has spent a fair amount of time examining rankings from an empirical perspective, I have come to one clear conclusion:   normative arguments, no matter how cogent and persuasive as a matter of principle, are simply irrelevant.  Rankings have whipsawed the legal academy because of institutional self-interest and a collective action problem. 

The chart below, which is from our Measuring Outcomes paper, is a strong piece of evidence for how we (the legal academy) have lost our way.   In 1997, U.S. News ranking added bar passage to its placement methodology.   This coincided with a movement toward higher bar cut scores.  (See Gary Rosin's Unpacking the Bar for the implications of this trend.)  Bar results have the advantage of being verifiable.  In contrast, that same year, U.S. News dropped the salary component from its methodology because of evidence that schools were routinely inflating their data. 

Andy and I had anecdotal evidence that schools responded to this pressure by increasing 1L attrition--i.e., imposing a strict curve that guarantees that a fixed percentage of students will flunk out.  Because 1L grades are strongly correlated with bar passage scores, this policy could boost bar passage.  According to data posted on the ABA website, between 1997 and 2004, law school attrition was up 20%.  But when the numbers were disaggregated, the following pattern emerged.