Empirical Legal Studies

ELS "has hit the world big time"

The Empirical Legal Studies Blog has now been identified by the Law & Society Association as the place to go to learn about the Empirical Legal Studies Movement.  See here.  (The Blog plug is on page 3, while the article about the rise of the ELS movement in law schools starts on page 2.)

Coping with the “Petty Tyranny” of IRBs

Edward Abbey quipped that “No tyranny is so irksome as petty tyranny.” But, rather than fighting or railing against the system, my final post this week invites discussion on coping with IRBs. My prior post, and Jack Katz’s, talked about claiming exemption and when it’s safe to not even file. This post focuses on the “minimal risk” threshold. Below it, protocols may qualify for expedited review and waiver of formal consent, but beware ye who pass above it.

Richard Campbell, member of a now-defunct advisory committee, offers this excellent analysis: the Common Rule defines minimal risk as “not greater than those ordinarily encountered in daily life.” This should mean the sort and magnitude one encounters working, shopping, driving, or having a doctor’s check-up, which includes temporary or low-grade harms such as stress, anger, embarrassment, stubbed toes, paper cuts, etc. Campbell notes that risk of even disastrous harm meets the “everyday” standard if it’s fairly rare, such as an auto accident.

Building on the transportation example, Bill Ford articulated the absurdities of regulating car driving risks as we do research risks. It’s easy enough to point out differences between the two that justify regulating one but not the other. But the key point is to match the type of regulation with the degree of risk. Ford’s absurd example of filing route plans before driving is in fact how general aviation is regulated by the FAA (more or less) - which makes sense for its degree of risk. Similarly, the Common Rule, as written, could limit and modulate IRB oversight appropriately - if only IRB administrators could be convinced. In response to Ron Wright, perhaps that’s what law professors should be trying to do. 

ECPR Summer School in Methods and Techniques

There is a new annual summer methods program to add to the mix, this year at the University of Ljubljana in Slovenia.  The dates for this year are August 7 – 18th.

Here is the pitch:

[The program] is targeted at advanced students (Master, beginning Ph.D) and junior researchers in political science and sociology, but students from all social and behavioural sciences are most welcome.

The aim of the summer school is to provide methods training opportunities in addition to MA and PhD curricula at European universities and to foster debate, interaction, and mutual understanding between various methodological perspectives at an early stage in academic careers. The curriculum covers introductory and intermediate courses in both quantitative and qualitative methods, and also features some plenary lectures and debates, so as to increase alertness of young researchers to other methodological perspectives and possibilities.

The 10 parallel courses offered in 2006 are:

1. Basic Statistics

2. Mathematical Concepts and Formal Modeling

3. Expert Interview

4. Cross-National Survey Design and Analysis

5. Multivariate Regression Analysis

6. Interpretive Methods

7. Quantitative Textual Analysis

8. Systematic Comparative Methods

9. Comparative Historical Analysis and Case Study Design

10. Network Analysis

And here is the link to the website and to the brochure.

Fun with Data II

I earlier posted about using data to write shorter, light-hearted or more accessible pieces.   If you're interested in another, I have written The Dubitante Opinion.  The Abstract:

"The 2004 Term of the United States Supreme Court resulted in 203 full opinions written, including 61 concurrences and 63 dissents. Judicial use of these three basic opinion types, the majority, concurring and dissenting opinions or variations thereof (e.g., concurring in part and dissenting in part) has been the norm in the Court and lower federal courts over recent decades. Yet, another type of opinion exists - the dubitante opinion. Judges rarely write dubitante opinions or use the term, and informal polling suggests not many legal scholars are aware of the practice. This short Essay endeavors to shed some light on the use of the term dubitante in judicial opinions and spark discussion as to the merits of the dubitante opinion - What is a dubitante opinion? When was the term first used, and how often is the term used? Who uses it and how? What are the consequences of its use? "

(FYI: The essay is only 7 pages long.)

Empirical Research on Administrative Law

See here.

An IRB Forum not on this Blog

Melissa Lauritch writes:

"I'd like to alert you to a great website called IRB Forum (http://www.irbforum.org), which basically covers everything IRB. You can access the discussion topics by registering on the site (it's free). You can sign up for the discussion e-mails (I must warn you though, you will be bombarded with e-mails) or you can simply browse or search the discussion topics online. I don't recall any discussion threads regarding law professors and IRBs, but I'm sure if you post a question you'll receive a number of helpful responses. Many of the frequent responders are extremely knowledgeable about IRB matters. "

Law School Affirmative Action Studies

Contributions to the growing empirical debate about the impact of law school affirmative action, triggered by Richard Sander's 2004 study (57 Stan L Rev 367), include a recent working paper by Lempert et al., which can be found here.  In it, the authors reanalyze the datasets used by Sander in his original study and respond to Sander's previous reply. For those interested in the necessary background reading, the Stan L Rev (57:6, May 2005) contains a comprehensive set of the initial responses and replies.

Putting Lawyers into the IRB Mix

Most legal academics at most law schools have never submitted research proposals to an IRB. Most, I would guess, have no idea that their research probably falls within the asserted jurisdiction of the IRB on their campus.

If this benign (?) neglect disappears over the next few years -- as I think it will at many universities -- what will be the effects on practices in law schools, and what will be the effects on IRB practices? I have a prediction on the second question.

Based on other interactions between the university administration and the law faculty, I'm betting on some healthy turmoil. When the pension fund administrators at the university make claims and suggestions about personal and university investments, the law faculty start talking about securities fraud. When the university administration wants to "manage" the message coming from the college radio station, the First Amendment mavens at the law school get lots of air time.

The point is that law faculties, as a whole, have a reputation on many campuses for pushing back against assertions of administrative authority. They are feisty, even contentious. Other groups of faculty might prove just as feisty, but the law faculty neutralize one favored administrative technique -- claims about the law -- that work as debate-stoppers when it comes to other faculty groups.

I think this dynamic will hold true to form when law faculty start interacting with IRBs in larger numbers. When an IRB makes an overly broad assertion of power, based on its reading of the regulations, a lawyer is more likely than a biologist or a sociologist to reject the current way of doing business and propose something different. If, as Jack Katz mentioned in his recent post, an IRB sees its mission as preventing research that could provoke some legal claim against the university, a legal researcher will be pretty vocal in pointing out the weaknesses of such a civil claim. And what does prevention of long-shot civil claims have to do with the Belmont principles?

If you chaired an IRB, wouldn't you quietly hope that legal academics remain in a gray area of possible non-compliance, just for the peace and quiet? 

Outside of Biomed Research, IRBs are Essentially Censorship Agencies

Like many ethnographers, I often do my research by putting myself in places and relating to people without a very clear idea as to what I’m looking for.  Even when I have a clear idea, like when I’m going to interview someone about their life in a neighborhood, I’ll see things on the way there and back that may be more valuable than what I find out when my questions get answered.  Much of my fieldwork becomes “data” in retrospect.  This makes it impossible to clear much of my research beforehand with an IRB.

To different degrees, many social researchers, not just anthropologists but political scientists and many others who would create “generalizable knowledge” and thus fall within IRB jurisdiction, work this way.  So do law professors, as entries on this blog have already acknowledged.

This means we are all vulnerable to attack for violating the rules in our universities or colleges. There are ways of limiting our vulnerability; I’ve detailed some in a paper that’s part of a collection currently being considered by an anthro journal.

Hiring PhDs

Jason mentioned here the possibility that more and more law schools will be hiring PhDs as law professors.  Conglomerate has discussed the possiblity that law schools might even begin to pay for PhDs for its faculty.  Amid all this discussion, though, I began wondering (as did Michael here and a commentator at Conglomerate here) whether it is advisable for law schools to seek out PhDs in their hiring and thought it might be worth bringing up again for more discussion.  Is hiring PhDs in law schools useful/advisable/desirable?  Are only certain law schools (e.g., those that are most likely to produce legal scholars rather than practicing attorneys) served by PhD faculty?  Does it make sense to hire PhDs who teach empirical legal studies at a law school primarily concerned with training lawyers?    Do lawyers need knowledge of empirical social science and how best to conduct it?  Just curious what the legal scholars' take is on this issue.  We PhDs are certainly noticing (not only because of the pay raise it would inevitably entail!).

Studying IRBs - Goldman, Katz, and Smith

Jerry Goldman and Martin Katz published a rare empirical study of IRBs in the Journal of the American Medical Association in 1982.  Here is a brief summary:

We submitted three flawed medical protocols with the expectations that the review boards would find the flaws evident in the proposals. But the boards, on the whole, did not find the flaws. Rather, they suggested changes that would, if approved, made bad research projects worse. This is the only major empirical study of IRBs and the findings have never been controverted with other evidence.

The paper can be found here.  Professor Goldman may have already started a replication of this study with Dr. Lewis Smith. (Thanks to Steve Wasby for calling my attention to this paper.)

The Empirical Assumptions in Environmental Law

In addition to courts and judges, my substantive area of interest is environmental law.  I've been working on two pieces that discuss the failure of the federal environmental regulatory regime and suggest some alternative solutions, namely a judicial interpretive solution and a rebirth of common law jurisprudence.  (If you're interested in more details, see one paper here or attend the Virginia Environmental Law Journal Symposium at UVa.)

Because the ELS Blog is always on the mind, it struck me how many empirical assumptions must be made with little or incomplete information when suggesting reforms in environmental law.  (Cf. Our earlier posts about whether data can resolve theoretical debates.)  For example:

• How will Congress react to broad/narrow environmental regulation?  Does it matter if this is a judicial or administrative conclusion?
• What are the potential hams to the environment and public health for failure to regulate if the statute is unclear as to whether regulation is required?
• Are slow CERCLA cleanups the result of administrative failures and red tape, or responsible time-consuming science?
• Do environmental impact statements correctly predict adverse environmental consequences?

These are only a few questions.  While we can point to certain case law, specific institutional reactions and case studies, I'm always in search of more data and trying to bring empiricism into environmental law.   Feel free to comment if you are aware of any good data sources related to environmental law.  Perhaps there are also papers outside of the legal academy (e.g., Law & Society) that empirically evaluate environmental law.

IRBs and Regulatory Theory

When I first saw an IRB at work, it reminded me immediately of a creature that is familiar to Administrative Law teachers: the agency that has incentives to ignore Type 2 errors in favor of Type 1 errors. The classic example is the Food and Drug Administration. The FDA knows that if it wrongly approves a dangerous drug for sale (Type 1 error), then when somebody dies after using the drug (say, using thalidomide as a sleep remedy for pregnant women), there is hell to pay. Public outrage, congressional hearings, you know the drill. On the other hand, if the FDA wrongly fails to approve -- or wrongly delays approval -- of a safe and useful drug (Type 2 error), the costs are more diffuse and less visible. There are exceptions (think of recent pressure on the FDA over AIDS treatment drugs), but the culture of the FDA pushes everyone in the agency to be most alert for Type 1 errors, even though both types of errors are costly to health.

The same incentives operate within a university IRB. The consequences of Type 1 errors -- such as wrongly approving harmful or non-complying research protocols -- are disastrous, including suspension or loss of funding for the entire institution. The IRB, on the other hand, tends to ignore the consequences of Type 2 errors, such as the delays in research, the research projects that never occur, the research that is needlessly scaled back in ambition.

This old chestnut from regulatory theory leaves a sobering message about IRB reform. It is not enough simply to point out the possible Type 2 errors and expect the IRB to account for them from now on. There are powerful reasons why Type 1 errors get the most attention. The strategies have to focus on ways to make Type 2 errors more visible and salient across many different bureaucratic cultures and across research proposals from many different parts of the academic world.

What we need are ideas that are universal amplifiers for Type 2 errors, loud enough to make stifling of research a routine worry for the IRB, on an equal footing with the Type 1 errors. Got any? 

Confessions of an Undocumented Human Subjects Researcher

I confess.  Like Jimmy Carter, I have lusted in my heart, and like most law professors, I have engaged in unprotected human subjects research (HSR).  Just last week, for instance, I called a lawyer and an insurance executive to get their views on a legal issue I was researching, and I might cite them in the article I’m writing.  Later, I wondered whether I should have had IRB permission first to do that. 

I expect that a good portion of this forum on IRBs will address overregulation, but I doubt we’ll be able to do much about that in the short term, so I would like to kick off a second line of discussion this week: practical tips for coping with IRB oversight.  To focus discussion, note that there are two regulatory thresholds: 1) is an activity non-exempt human subjects research (HSR) to begin with; and 2) if so, does it pose more than minimal risk?  This post focuses on the first of these two thresholds. 

As summarized in the IRB FAQs that Ron Wright and I posted several weeks ago, there are several bases to claim that interaction with, or data about, living humans in the course of legal scholarship is not a covered activity (not identifiable, not systematic, not private, etc.).  Naturally, those concepts have room for interpretation.  The hitch is that institutions expect researchers to notify the IRB so it can have the final say on whether the exclusionary concepts apply. But, that quickly leads to innumerable absurdities.  Every hour of every working day, I do things that are not HSR.  Which of those things do I need to tell the IRB about, simply to confirm they aren’t non-exempt HSR?  Do we also need to define a grey zone threshold for when an activity MIGHT be interpreted as covered HSR by an IRB?

Not necessarily.  I believe this threshold decision is one that researchers should be able to make on their own in good faith, albeit at their own risk of being wrong.  If I’m not seeking external funding, and I’m not publishing the research in journals that routinely deal with HSR, then it’s not likely that anyone will require me to verify the absence of HSR.  And, if in fact I haven’t done any HSR, then have I done anything wrong by not clearing the activity first with the IRB?   If not, then maybe I only need to seek IRB approval if I’m not sure about the activity being HSR, OR if I anticipate having to verify to an outsider (journal or funding agency) that it isn’t covered HSR.  Am I wrong about this?  How much trouble could I or the institution get into for doing things that aren’t covered HSR without seeking IRB approval first?

Addendum:  Jack Katz’s related post refers to an excellent paper of his (available soon here, and in press at the American Ethnologist), that develops two key points in detail:  1) IRBs are not required to review research that is not federally funded.  Although most Universities require this, some prominent ones are starting to step back from that.  2) Some types of research involving humans are not sufficiently “systematic” to meet the Common Rule’s threshold definition.  It’s a really good paper; you should read it. 

Thank You

This is a belated thank you to the creators of the ELS for the kind invitation to serve as a guest blogger. And, to those who offered their comments in response to my posts last week. They were ALL thoughtful and very useful for me (and I hope readers of the blog as well).

Human Research Protection Programs (HRPP) Blog

The HRPP Blog offers a perspective on IRBs from someone with extensive experience as an IRB professional and consultant.  There are only a small number of posts so far, but he does comment on the Center for Advanced Study's "Mission Creep" white paper in the most recent one.

Please Have Your Consent Forms Ready

Is change afoot in the regulation of social science research by IRBs, especially low risk research involving surveys, interviews, and the use of already existing data?  One source of change could be the intensifying scrutiny of IRBs. In April 2003, the Center for Advanced Study held a conference on IRBs and published a white paper on the topic, Improving the System for Protecting Human Subjects: Counteracting IRB “Mission Creep.”  Next month at Northwestern University, Philip Hamburger (Columbia) and James Lindgren (Northwestern) are holding a conference on the First Amendment implications of IRB regulations.  If academic conferences can provoke change anywhere, maybe it’s in academia.  Another possible source of change is the increasing interest of legal scholars in empirical research, which means more IRB oversight of their work.  Presumably, regulating legal scholars increases the likelihood of an unhappy researcher filing a lawsuit against an IRB -- and perhaps making the arguments generated at these conferences to a court.  Whether changes are likely and what changes are desirable are two of the topics up for grabs this week.

During this second Blog Forum, four guest bloggers will offer their perspectives on IRBs.  Kristina Gunsalus of the University of Illinois’ Office of University Counsel and the College of Law (Urbana-Champaign) has written widely on the topic of IRBs.  She is the primary author of the Center for Advanced Study’s white paper.  Mark Hall and Ron Wright of Wake Forest University’s School of Law have experience from the law school side of dealing with IRBs.  Professor Hall specializes in health care law and policy.  Professor Wright specializes in criminal justice and administrative law.  Jack Katz of U.C.L.A.’s Sociology Department is a participant in the upcoming conference at Northwestern University.  He specializes in social psychology; ethnographic methods; urban communities; and crime, law and deviance.

Next Week: A Blog Forum on Institutional Review Boards (IRBs)

Suppose long ago the local Departments of Motor Vehicles, perhaps under the auspices of the Federal Motor Carrier Safety Administration, started regulating the transportation of hazardous chemicals on public roads. They would inquire about destinations, routes, and safety protocols. Since the risks from these materials may be significant, this oversight might be a good idea. But at some point, assume they decided to monitor pretty much all travel on the public roads: No one should travel anywhere unless independent review concludes that the risks are reasonable in relation to the potential benefits. Cf. Nat’l Bioethics Advisory Commission, I Ethical and Policy Issues in Research Involving Human Participants ii (August 2001). In other words, no one should be judging whether to go anywhere for him or herself, until after someone else has decided it’s okay. See C. K. Gunsalus, The Nanny State Meets the Inner Lawyer: Overregulating While Underprotecting Human Participants in Research, 14 Ethics & Behavior 369-382 (2004).

At least for now in this hypothetical world (FN1), travel plans on private roads are not covered at all, and many travel plans on public roads are exempt from the DMV’s oversight. If it involves travel on public roads, however, only the DMV can judge it as exempt. Some common plans are pre-approved as exempt, e.g., travel to local grocery stores along certain routes. But you probably still need to send an e-mail to your DMV notifying them of your intent to use these pre-approved plans. They’d like to know what you’re up to.

Measurement

The utility of the empirical verification of theories of judicial decision making is dependent on developing sound measures for key theoretical concepts. This may seem to be so obvious a statement as to be trivial but the truth of the matter is that most scholars most of the time are not appropriately self-conscious about the issue of measurement. Certainly we worry about things like inter-coder reliability (something Jason raised in a post on February 28). Further, there are, of course, all sorts of debates over how to measure the ideology of judges (a point that is relevant for Jason’s post on March 2 regarding the Supreme Court Ideology Project). (See, for example, Christopher Zorn and Gregory Caldeira’s paper “Bias and Heterogeneity in a Media-Based Measure of Supreme Court Preferences” and Epstein et al.’s paper “The Judicial Common Space.”) And, how we measure “the law” is something that is increasingly preoccupying scholars (as Sara’s post of March 13 illustrates).

A recent conversation with Paul Collins about brought the issue of measurement to my mind again. In particular, Paul argues that a common measure of salience—the presence of amicus curiae briefs—is really a measure of complexity rather than salience (in that those briefs have the potential to bring to the fore heretofore unconsidered policy or legal dimensions). In one sense, amicus curiae briefs certainly are indicators of salience, at least to the interest groups or other third parties who are filing them. But, as Sara Benesh and Harold Spaeth have argued in a 2001 American Political Science Association conference paper, key to determining an appropriate measure of salience is thinking about it in terms of the question: to whom is it salient? In their January 2000 article in the American Journal of Political Science, Lee Epstein and Jeffrey A. Segal offered a measure of salience based on media coverage. Saul Brenner and Ted Arrington, in an unpublished but widely circulated manuscript, evaluated the virtues and vices of that measure, concluding that a hybrid measure relying both on New York Times coverage and the list of major cases compiled by Congressional Quarterly was preferable. In light of the Epstein/Segal and Brenner/Arrington measures, Sara and Harold developed a prototype measure based on the syllabus of a case. Their argument is that the syllabus, though prepared by the Reporter of Decisions, Deputy Reporter of Decisions, and Assistant Reporter of Decisions, must be approved by the majority opinion author and, hence, is a better basis on which to assess salience to the justices themselves.

The lesson I draw from this work collectively is that we need to think carefully not only about the reliability of our measures but their validity, too. It may not be a particularly novel or exciting lesson but it is worth bearing in mind.

MPSA Conference Program Available

The Midwest Political Science Association Conference Program is now available.

"Bringing Methods to Our Madness"

There were earlier suggestions that we should have a new, more-welcoming motto.  I also received an email today where, after saying how she enjoyed our blog, the emailer wrote:  "[T]his is the worst motto I have ever heard. These guys are brilliant but tone deaf. "  Thus, I am pleased to announce our new motto:

www.elsblog.org -- Bringing Methods to Our Madness

 

Harcourt and Ludwig on Broken Windows

In the new issue of the University of Chicago Law Review (Winter 2006), Bernard Harcourt (Chicago) and Jens Ludwig (Georgetown) revisit James Q. Wilson's and George Kelling's classic Broken Windows argument. Here is the abstract:

In 1982, James Q. Wilson and George Kelling suggested in an influential article in the Atlantic Monthly that targeting minor disorder could help reduce more serious crime. More than twenty years later, the three most populous cities in the United States -- New York, Chicago, and, most recently, Los Angeles -- have all adopted at least some aspect of Wilson and Kelling’s theory, primarily through more aggressive enforcement of minor misdemeanor laws. Remarkably little, though, is currently known about the effect of broken windows policing on crime.

Long Voting Lines and How to Write a Publishable Paper

Two interesting articles in the new "PS: Political Science and Politics":

Long Lines, Voting Machine Availability, and Turnout: The Case of Franklin County, Ohio in the 2004 Presidential Election by Benjamin Highton

Publication, Publication by Gary King

Comparative and Transnational Courts

This semester I am teaching a seminar on judicial politics and behavior. It is, for the most part, one of the standard versions taught by people who primarly teach about and study American courts. Much too narrow, of course, given that it is a one semester class and structured largely by my own intellectual interests, background, and training. (I say this in the interests of full disclosure. My guess is that I don't do any better or any worse than most faculty in this regard.) But, it is a small seminar (with four advanced undergraduates, two first-year graduate students, and two second-year graduate students). The interests of the seminar participants are quite diverse. For example, two of the graduate students are primarily interested in comparative legislative institutions, one is interested in international relations, and one is interested in judicial politics, primarily in the American context but increasingly from a cross-national perspective). Accordingly, I modified my "standard" reading list to enhance the comparative courts readings and to add a section on transnational courts.

Our seminar discussions have given rise to several queries that I would like to pose here.

First, when is it appropriate to examine theories of judicial behavior in the context of,  for example, the U.S. Supreme Court or state courts of last resort and when is it more appropriate to do so in a comparative context (whether "comparative" is read as meaning comparison across several countries or as a single country study as long as that county is not the U.S.)?

Second, why is the literature on comparative courts (seemingly?) focused single-mindedly on the theory of separation-of-powers? Is that the only "judicial" behavior of interest in a comparative context?

Third, where do transnational courts (e.g., European Court of Justice) or quasi-courts (e.g., NAFTA's binational panels) fit in the study of judicial politics and behavior (or law & courts or public law or whatever your favorite rubric is)?

Data Availability: Protocols & Emerging Norms

As legal scholarship becomes increasingly empirical, legal scholars need to become more aware of and sensitive to relevant scholarly norms that inform empirical work.  One underexamined norm relates to expectations concerning data availability and the facilitation of replication.  Because replication is central to the empirical scholarship enterprise scholars owe some duty to facilitate replication by others.  Such a duty, of course, necessarily implicates access to data (and, as I discuss below, possibly to coding as well).  The nature, extent, and contour of that duty, however, are far from clear and, indeed, variation distinguishes various disciplines.  Two common scenarios illustrate some of the complexities and nuances.

One scenario involves publicly available data, such as those managed and archived by ICPSR at Michigan.  For legal scholars that use such data in published work, one obvious expectation (indeed, requirement) is to identify the specific dataset by its ICPSR number in a footnote (or table note).  However, is mere dataset identification enough?  For example, because some amount of data preparation and manipulation (e.g., collapsing, filtering, re-coding) is almost inevitable, should legal scholars also be expected to make available their coding?  Similarly, should table-specific coding be made available by authors?  To do so would reduce the burden on subsequent scholars enormously and facilitate replication, follow-up analyses, etc.

A second scenario is even more delicate as it involves original datasets generated by scholars and not publicly archived.  To be sure, it is difficult to overstate the effort necessary to develop a first-class dataset.  In light of a scholar's often consider investment of sweat equity and because quality datasets frequently support multiple articles, what obligations attach to scholars in terms of facilitating replication efforts by others?  On the one hand, a mechanical requirement to release the complete dataset, including data not yet published on (a "total disclosure duty" position), would likely deter dataset building efforts, at least at the margins.  At the other extreme, general scholarly norms would assuredly resist a "no duty to disclose" position as some amount of disclosure and data availability are necessary so that findings can be vetted and knowledge advanced.  To be sure, considerable middle ground separates these two polar positions.

I welcome thoughts and perspectives on how empirical legal scholars should resolve these (and related) issues and navigate through this uncertain and evolving terrain.

Why do Judges Dissent? - Part I

What factors influence judicial agreement versus dissensus--the choice to write a separate dissenting or concurring opinion?  A number of variables may play a role: Ideology, Collegiality, Trial Court experience (of interest to Martinek's earlier post), and the high costs of dissenting due to increased workload.  (See, e.g., Edwards 1998; Czarnezki/Ford forthcoming 2006; Benesh/Spaeth forthcoming; Hettinger/Lindquist/Martinek 2003).  While legal factors have not been shown to play a major role, there may be consequences between using directional vote versus agreement as the dependent variable.

In a work-in-progress with Bill Ford, we examine aggregate disagreement rates on four U.S. Courts of Appeals in an attempt to explain dissensus as a product of these above variables and more.   Below is a graph showing the overall dissent rates in the fours circuits since 1946.  Preliminary results as to what variables correlate with these rates are pending... stay tuned for Part II of this post.  (And if interested, the data will presented Sunday morning at the Law & Society conference in Baltimore for the panel "How Judges Make Decisions.")

2005 Federal Court Caseload Stats Now Available

See here.

Small Group Theory and Judging

Three and four decades ago, students of the courts such as Walter Murphy and S. Sidney Ulmer paid particular attention to how the group context shaped judicial behavior and court outcomes, drawing on the political and social psychology research of the day. However, contemporary scholars pay attention to the group context for appellate court decision making only to the extent that it provides an opportunity for strategic behavior. But judges, as human beings, are profoundly affected by the identities of those with whom they make decisions and the norms of the environments within which they make those decisions. I would argue that not all of those concerns are, strictly speaking, strategic in nature.

To illustrate, consider designated judges. The chief judge of a circuit has the authority to temporarily assign (designate) a district court judge from his circuit to serve as part of an appellate panel. According to data available from the Administrative Office of the Courts, designated judges account for upwards of twenty percent of all case participations in the United States Courts of Appeals. Though some of these designated judges are circuit court judges serving from other circuits, the majority are district court judges. A basic question is whether these designated district court judges are fungible with their circuit court colleagues or if their status as district court judges affects their ability to be fully equal partners in the deliberative process. Work that appears in Law & Society Review in 2001 by Brudney and Ditslear ("Designated Diffidence: District Court Judges on the Courts of Appeals") provides some insight into this question, as does earlier work by Thomas G. Walker that appeared in the American Journal of Political Science in 1973 ("Behavioral Tendencies in the Three-Judge District Court).

Think, too, about how the presence of a designated judge may affect the behavior of the regularly sitting appeals court judges with whom he sits. Much of the workload of the United States Courts of Appeals consists of cases appealed from the federal district courts. Do the regularly sitting appeals court judges reviewing a district court ruling in the company of a designated district court judge approach that review in the same way or do they suppress the error correction function inherent in the appellate court process in deference to their district court colleague?

I am intrigued by this line of inquiry precisely because it represents an empirical question the answer to which has important implications for a normative concern. (Or, if you like, a normative question to which empirical evidence can be brought to bear.)

A New Member of the ELS Blog

I'm pleased to announce that Sara Benesh has agreed to become a permanent Editor of the Empirical Legal Studies Blog.  As a Guest Blogger, Sara discussed the normative implications of empirical work, how to operationalize "the law", and the Supreme Court database. 

Sara is an Assistant Professor of Political Science at the University of Wisconsin-Milwaukee.  She received her PhD in 1999 from Michigan State University, held a tenure-track position at the University of New Orleans for two years, and joined the faculty at UWM in 2001.  Her research focuses on the relationship between the Supreme Court and the U.S. Courts of Appeals, and she is the author of “The U.S. Court of Appeals and the Law of Confessions: Perspectives on the Hierarchy of Justice” (LFB Scholarly) and co-author of “The Supreme in the American Legal System” (with Jeffrey Segal and Harold Spaeth, Cambridge University Press). She is also Co-PI (with Harold Spaeth) on the National Science Foundation-funded “Justice-Centered Databases,” a revision to the celebrated Spaeth Databases.  Sara teaches courses in civil rights and civil liberties, judicial behavior, and political methodology at both the graduate and undergraduate levels. 

Law and Courts Section Awards

Mark Graber, the chair of the Law and Courts section of the American Political Science Association, has announced the recipients of two awards:

The 2006 American Judicature Society Award for "the best paper on law and courts presented at the previous year's annual meetings" goes to Kevin McGuire and Georg Vanberg for their paper, "Mapping the Policies of the U.S. Supreme Court."

The 2006 C. Herman Pritchett Award for “the best book on law and courts written by a political scientist and published the previous year” goes to Peter H. Russell for his book, Recognizing Aboriginal Title: The Mabo Case and Indigenous Resistance to English-Settler Colonialism (University of Toronto 2005).

Interesting Thoughts on Judicial Tenure

See here.

How Do We Know?

Over the past week or so I have been slogging through a stack of texts I am considering adopting for the introductory course on research methods (which includes some epistemology of science, research design, basic statistics) taught in my department each fall for first-year graduate students. This is the first time I will be teaching this course so the choice of text(s) looms particularly large. And, to complicate matters, we have recently added an an introduction to formal modeling course in the second semester for our graduate students, which they will take in addition to a course on statistics (beyond that which is covered in the first-semester course). So, I am very cognizant that what I cover in the first-semester course needs to give students preparation for not one but two courses they will take in their second semester. (I expect that neither of my colleagues who will be teaching those second-semester courses will find the students as prepared as they would like them to be but, being good natured folks, I am sure they will be patient with both the students and me.) I think that the sort of issues with which I am grappling in (re)designing this course are issues that speak more broadly to what it is that those interested in empirical legal studies care about. Let me (hopefully) provoke a discussion by offering a two guiding principles on which I am relying:

Contest: A Tournament of Party Strategies

James Fowler (U.C. Davis) and Michael Laver (NYU) are sponsoring a spatial modeling contest involving party electoral strategies. The prize for the most successful strategy is $1,000.  Entries must be received by April 15, 2006.

New Syllabus - Qualitative Research Methods

Recent guest blogger Artemus Ward has added his Qualitative Research Methods syllabus to the Society for Political Methodology's syllabus collection.

Guest Blogger: Wendy Martinek

Introducing our Next Guest Blogger: Professor Wendy L. Martinek (Binghamton) specializes in the study of judicial politics, with a particular interest in the judicial selection politics at the federal level and decision making in both the United States Courts of Appeals and state courts of last resort. Her current research program includes a project examining amicus curiae in the United States Courts of Appeals.

In addition to her focus on judicial politics, Professor Martinek has complementary interests in American political institutions, interest groups, state politics, and political methodology. Her research has been published in the American Journal of Political Science, the Journal of Politics, Social Science Quarterly, American Politics Review, Party Politics, and Justice System Journal. Her book, Judging on a Collegial Court: Influences on Federal Appellate Decision Making, with Virginia A. Hettinger and Stefanie A. Lindquist is forthcoming with the University of Virginia Press.

Dining with Chief Justice Burger

If you liked our blog forum about researching law clerks and law clerk confidentiality, Joseph Zengerle, who clerked for Chief Justice Burger in October Term 1973, has written "Changing of the Chiefs."  9 Green Bag 2d 175.  (Note: The Green Bag is one of the few journals worth reading back to front.)   The article is not yet up on the Green Bag website, but I suspect it will be soon.  The article briefly discusses Zengerle's deliberations with CJ Burger regarding Cleveland Bd. of Ed. v. LaFleur, 414 U.S. 632 (1974), and spends significant time describing the annual dinner of the WEB Fete Society.

"A Gentle Introduction to Stata"

Adding to our discussion of useful statistical texts for students and new users, Alan Acock has written A Gentle Introduction to Stata, being sold as "an ideal book for students and for those with experience with other software packages but new to Stata."

A New ELS Blog Motto

Dave Hoffman at Concurring Opinions writes :

"The ELSblog has been on fire recently, including posts on law clerk loyalty and legal research; IRB Boards; the relationship between empirical work and theory; and a little dip into the old student-review incompetence debate. Based on quality-of-post, the blog should be getting more hits than it does. I'm thinking that their current motto ("advancing productive interdisciplinary discourse among empirical legal scholars") is a little unwelcoming. But otherwise, they are doing neat work. Go check 'em out!"

First Dave, thanks for the kind words!  Second, I'm now taking suggestions:  What should our new motto be?  I look forward to comments.

Full Circle

And so we've come full circle.  I began the week asking how best to operationalize the law.  My most recent post on the Attitudinal Model generated much discussion over how much that explanation of decision making leaves wanting.  So, what's the alternative?  Harold Spaeth is himself the first to say (at least to me) that if someone can demonstrate a "better" way to explain Supreme Court voting behavior, he's all ears.  That means, of course, that he's looking for an empirical, falsifiable, hypothesis-based demonstration that there is something else that systematically, significantly, and substantively affects Supreme Court decision making.  And so I ask again, what is the alternative?  Is law a "nuisance" as Zorn describes it (drinking wine and crashing into tables, no less!!), or are there systematic components to the law that we might operationalize and include in our models of Supreme Court decision making?  If attitudes are the best predictor of votes (regardless of how much they actually are able to predict, alone or with facts), then isn't that still a pretty major and useful contribution?  And does anyone have something in mind that does better?

So, coming full circle is a nice way to end my week as "Guest Blogger!"  Thanks for your interest and attention!

Normative Implications of the Attitudinal Model

I am a social scientist.  Therefore, whenever anyone says the word "normative," I squirm.  I'll admit it.  But I have been thinking, lately, about some of the normative implications of widespread knowledge of the attitudinal model of judicial decision making.  The Law and Courts Listserv has been exchanging instances of hostility toward judges:  Justice Ginsburg's internet death threats for reliance on foreign sources of law; the movement to seize Justice Souter's property in New Hampshire; the new "J.A.I.L 4 Judges" Amendment in South Dakota.  Add to that list Pat Robertson's comments on This Week that activist judges are as bad as the 9/11 terrorists; the Senate Majority Leader's statements over the Shiavo case regarding Congress' power over the federal courts; Sensennbrenner's letter to the 7th Circuit on a sentencing decision; and on and on.  (The Brennan Center, of course, does an excellent job of tracking these threats.)  What is going on in terms of our confidence in our judiciary and over the presumed benefits of judicial independence? 

Attitudinalism & Federalism

Attitudinalists (and others) might be interested in a new working paper by Mike Dorf (Columbia), entitled: "Whose Ox is Being Gored? When Attitudinalism Meets Federalism".  An excerpt from his abstract follows:

“Empirical research indicates that factors such as an individual Justice's general political ideology play a substantial role in the decision of Supreme Court cases.  Although this pattern holds in federalism cases, views about the proper allocation of authority between the state and federal governments - independent of whether the particular outcome in any given case is "liberal" or "conservative" - can sometimes be decisive, as demonstrated by the 2005 decision in Gonzales v. Raich, in which "conservative" Justices voted to invalidate a strict federal drug provision in light of California's legalization of medical marijuana, and "liberal" Justices voted to uphold the federal law.”

New Blog Forums

We have added two new blog forums to the schedule. The first is on the topic of Institutional Review Boards, a.k.a. Human Subjects. It’s scheduled for March 27-31, the week before the IRB conference at Northwestern University. The second is on the New Legal Realism Project and is scheduled for June 19-23.

The current participants are listed on the right. More names to come.

Which Supreme Court Book to Assign?

The book The Supreme Court in the American Legal System (Segal/Spaeth/Benesh 2005) is reviewed by Richard Pacelle (Georgia Southern) in the recent Perspectives on Politics (Vol. 4, No. 1, Mar. 2006).  While the short review recognizes that the book "is designed to be a less methodologically sophisticated analysis," the review questions the book's usefulness--"this will be a book in search of a market"--because Pacelle "suspect[s] instructors will opt for a more nuanced treatment of judicial decision making."  Pacelle's hypothesis may be correct; but perhaps the book might also be filling a gap in the market.  For example, to my knowledge, it is the only text on judicial process/behavior that includes a website with empirical data and commands (in SPSS) to let students replicate the analyses presented in the book.

It seems that judicial politics/judicial decision-making/American legal system/Supreme Court seminars involve professors and students with a wide range of methodological interest and expertise.  While the ELS Blog discussed book selection in the classroom before, no suggestions were made about what judicial behavior books should be used for different types of courses with students and professors of varying interests and training, e.g., graduate versus undergraduate students; students with versus without statistical training.  (A hierarchy was successfully created for statistical software.)   I've seen Robert McCloskey's The American Supreme Court assigned for both undergraduate non-empirical courses and law school empirical seminars on the courts, with most graduate judicial politics seminars in law and the social sciences assigning The Supreme Court and the Attitudinal Model Revisited. 

What is the hierarchy of foundational judicial decision-making texts in terms of methodological sophistication?  Or, more specifically, what should one assign law students (or upper level undergraduates) with no previous statistics training in a judicial decision-making seminar?

Spaeth's Supreme Court Database

For those of you interested in empirical research on the U.S. Supreme Court, I'd like to call your attention to a set of articles in the most recent Law & Courts that may be of use.  Kirk Randazzo's discussion of the Ulmer Project, my article entitled "Becoming an Intelligent User of the Spaeth Databases" and Paul Collins' article entitled "Transforming the Original U.S. Supreme Court Judicial Database" may all be of use, especially to new users.  I'm happy to take questions about the databases as well, either here on the blog or via email ([email protected]).

Law School Rankings and More from the Law & Society Review

The latest issue of the Law & Society Review (Vol. 40, No. 1) has some interesting articles:

Do Rankings Matter? The Effects of U.S. News & World Report Rankings on the Admissions Process of Law Schools by Michael Sauder & Ryon Lancaster

• "Using data for U.S. law schools from 1996 to 2003, we find that schools' rankings have significant effects on both the decisions of prospective students and the decisions schools make in the admissions process. In addition, we present evidence that the rankings can become a self-fulfilling prophecy for some schools, as the effects of rank described above alter the profile of their student bodies, affecting their future rank. Cumulatively, these findings suggest that the rankings help create rather than simply reflect differences among law schools through the magnification of the small, and statistically random, distinctions produced by the measurement apparatus."

The Influence of Jurisprudential Considerations on Supreme Court Decisionmaking: A Study of Conflict Cases by Stefanie A. Lindquist & David E. Klein

• "Our analyses reveal that the justices are (1) more likely to follow the reasoning process adopted by the majority of circuits involved in the conflict, (2) less likely to adopt the conflict position marred by contrary dissents and concurrences in the circuit court opinions, and (3) more likely to adopt the conflict position endorsed by prestigious circuit court judges. Our findings suggest that jurisprudential considerations, as well as attitudinal concerns, affect the justices' decisionmaking processes in a substantial minority of cases."

Understanding Judicial Hierarchy: Reversals and the Behavior of Intermediate Appellate Judges by Kevin M. Scott

• "One of the central controversies in the judicial behavior literature is the extent to which judges' ability to act according to their ideological preferences is affected by their location in the judicial hierarchy. Judges on intermediate appellate courts have different decisionmaking environments than high court judges. As a result, the goals of lower appellate court judges may differ from those of their superiors: the quest for legal accuracy may compete with the desire to pursue policy preferences."

Empirical Study of Brownfields and Residential Property Values

As an empiricist, who also teaches environmental law and property, I'm announcing that Dennis Kaufman and Norman Cloutier have posted "The Impact of Small Brownfields and Greenspaces on Residential Property Values" on SSRN. 

Operationalizing "the Law"

One would hope that the now-more-common collaboration between Political Scientists and Law Professors would result in the increased ability of Political Scientists to "take the law seriously."  Empirical Legal Studies should certainly have as at least one of its foci the empirical study of the extent to which law matters or the extent to which law constrains judges.  Some political science scholarship and some legal scholarship has begun to address this issue.  I'd like to point out what we know in order that we might fruitfully discuss where to go from here.  How can one take something as nebulous and multifaceted as "the law" and operationalize it so that its effects might be quantitively studied?  While one anonymous reviewer has suggested to me that the debate between "the legal model" and "the attitudinal model" is a tired one, it is certainly not the case, it seems to me, that we are even close to knowing all we need to know about the effects of the law on judicial decision making at all levels.  The following sketches out what some political scientists have offered as ways in which to test the influence of law.  How have they done?  How could we do better?

MPSA Program

The Program Overview and Preliminary Conference Schedule for the Midwest Political Science Association Conference (Apr. 20-23) is now available.  There are a number of panels under the general topics of Judicial Politics, Public Law, Legislative Politics, Presidency and Executive Politics, Methodology, Political Parties and Interest Groups, Voting Behavior, and much, much more.

Med Mal Database

I've noticed a recent spate of empirical papers assessing med mal, especially studies endeavoring to examine potential impacts flowing from various state tort reforms, drawing on the National Practitioner Data Bank Public Use Data File ("NPDB").  The Department of Health and Human Services ("HHS") maintains this data file in accordance with the Health Care Quality Improvement Act of 1986.  Since September 1, 1990, laws require that (with some exceptions) all medical malpractice payments must be reported to HHS within 30 days of payment.  Like any other database, however, NPDB is far from perfect.  For example, NPDB records do not satisfactorily include the individual characteristics of each case (e.g., injury severity, plaintiff age, physician specialty, etc. are either not or poorly reported).  Data file limitations aside, those interested in this literature can get more info on the NPDB here.

Law & Society Conference

The Law & Society Association Annual Meeting (Baltimore, July 6-9) Preliminary Program is now available.  Some of the following panels might be of interest to our readers:

• Bureau of Justice Statistics:  Criminal Court Statistics Programs
• So What Happens to Civil Cases on Appeal?  Profiles from the 2001 Civil Justice Survey of State Courts
• Cutting Edge Jury Research
  
• Author Meets Reader--Sorcerers’ Apprentices: 100 Years of Law Clerks at the U.S. Supreme Court, by Artemus Ward and David Weiden
• Courts in Context: Politics and Expertise
• Studying Variations in Leadership and Participation: Judges and Juries
• How Judges Make Decisions