Measuring Case Complexity

Thanks a lot to Jason and the other editors for inviting me to blog this week. I plan to spend my time here tapping the collective wisdom of the blogosphere about what I view as a number of important questions. I'll also spend some time blogging about various tools we should all be using in our empirical work.

I'll start with this. One of things political scientists oftentimes include in statistical models of judicial decision making is a measure of "case complexity." In the Supreme Court context, the standard measure, as coded by Harold Spaeth, is to label a case "complex" if there was more than one legal issue or provision at play in the case. This may be not be an accurate measure of complexity. Many cases involve, say, a first amendment free speech issue as well as a due process or establishment of religion issue. Is this more complex than a tax case? Securities? Telecommunications? Surely not.

Any thoughts about how to systematically measure case complexity? Perhaps looking at the number of issues cited or the level of factual complexity might help. How could we do better?

Comments

[David Zaring]

I'm very late on this, but I'll note that the courts of appeals clerks offices classify some cases as complex when they set up the schedule. So that measure could be used - it, however, covers a rather small number of cases.

Since most clerks find cases that arise from complex statutory schemes to be the most confusing, it wouldn't be ridiculous to evaluate Supreme Court complexity by checking the circuit from which the case came. DC or Federal = complex.

[Margo Schlanger]

It's not quite what Andrew is getting at, but in (what I hope will some day be more plentiful) studies of trial courts one can come up with a fairly objective and useful measure of complexity, once a case is over, by counting the number of items in a docket sheet. I've used this measure myself, and had students use it as well.

For appeals, seems to me that "complexity" can mean a bunch of different things, and while each has proxies, those proxies are unlikely to cover more than one or two definitions. So here are a few:

a) Legal difficulty: Esp. in statutory cases, the question could be, how many different statutes, or statutory provisions, are involved. Part of what makes ERISA cases so complex is the interplay of the statute and its various exclusions etc. Same thing with tax cases.

b) Multiple and separable issues.

c) Multiple potential outcomes.

d) Multiple actual outcomes (fractured opinions are more complex than non-fractured ones)

I'm sure there are more.

Again, which one you're interested in will matter a lot to what measure you use.

[Jeff Yates]

A possible wrinkle to the question of issue complexity - is there an objective standard for issue complexity or does it vary by the person(s) addressing the issue? In other words, is it about the potential learning curve? A justice well versed from private practice on patent law (or ERISA, or fill in the blank here) might find such a case relatively easy compared to, say, a criminal procedure case. But other justices would find it to be a complex issue. Justices may bring to the Court certain areas of expertise. Perhaps some issue areas are better represented on the Court (background expertise) than others.

[Geoff McGovern]

How to define complexity? Pennoyer v. Neff.

Frank's warnings about operationalization are keen and exactly what I suspect Andrew was driving at. There are likely to be numerous components to complexity. Which we choose depends on the questions we ask. A few possibilities: number of subject areas; number of questions presented in litigants' briefs; truly novel questions of law; labrynthal factual situations (those last two are hard to objectively articulate).

I recall Justice O'Connor remarking that the Court does not take easy cases, which comment would imply that the entire SCOTUS docket is filled by cases in some way complex. So, I tend to think using case complexity as a variable is a complete misnomer, though the actual data are helpful in our models.

[Ryan Owens]

I think Professor Cross raises an interesting point here. Perhaps we need to be more specific with what we mean by ``complexity.''

After all, complexity might come in (at least) two forms: (1) Issue complexity, as per the Lynch article; or (2) Complexity in decision making (i.e. with numerous legal issues in a case, justices might be able to engage in heresthetic to manipulate issue dimensions and forge winning coalitions out of otherwise losing ones). For more on this, see Riker (1986) and Epstein and Shvetsova (2002)

[frank cross]

I think it depends on what you are trying to capture with complexity. Multiple issues is obviously one relevant measure, if only because it raises bargaining and Condorcet issues

I don't think it captures complexity of the legal issues, in the sense of intricacy or difficulty. But I can't think of how to measure that.

[Michael Heise]

I agree; Andrew raises an interesting and vexingly complicated question. A few years ago I worked with a somewhat unique database put together by the good folks at NCSC and originally designed to help analyze the hung jury problem. On the theory that case complexity might influence the probability of a hung jury in criminal cases, the dataset included measures of "case complexity" (scored on a 7-point Likert scale) drawn from judge, jury, and participating (lead) attorney surveys. The study involved approximately 355 criminal trials in four separate jurisdictions and 1,300 participants. I sought to exploit the circumstance of three sets of independent actors assessing the complexity of the identical case by looking at whether explanations for what made a case complex varied (they did) and possible reasons for this variation.

[Ryan Owens]

This is an interesting question. In a recent article, Kelly Lynch, an undergrad from the University of Penn, interviewed a host of former Supreme Court Law Clerks. During those interviews, a majority of clerks told her that amicus briefs were helpful in certain complex cases.

The clerks specifically cited Railroad preemption, marine labor, water rights, ERISA, tax, patent, trademark, telecommunications, and immigration cases as those in which the Court needed more information. ``We didn't know anything about that [particular issue] and there were billions of dollars at stake'' (Lynch 2004: 41). In short, the Court had a difficult time understanding the facts of some of these disputes as well as the possible consequences of its rulings. It seems as though these should be the types of factors we look for when determining whether a case is ``complex.''

On the other hand, a sizable number of clerks argued that additional information was actually unhelpful in constitutional law cases. Since these constitutional law cases frequently involve more than one legal issue, our current measures of complexity may not be correct. Perhaps an issue specific focus is appropriate...

See Kelly Lynch. ``Best Friends? Supreme Court Law Clerks on Effective Amicus Curiae Briefs.'' 20 Journal of Law and Politics 33 (2004).