Guest Blogger: Kevin McGuire

This week's guest blogger is Kevin McGuire, an Associate Professor in Political Science at North Carolina at Chapel Hill.  Kevin teaches courses on the U.S. Supreme  Court and American constitutional law.  His current research interests include the influence of public opinion on the justices’ decisions, the content analysis of the Court’s written opinions, measuring the ideology of the Court’s case outcomes over time, and compliance with judicial policy.

Comments

[Dwight Hines]

What about caseload statistics for courts and the recent changes in crime rates? This article was written in 1991, is it still a good approach to avoidiing misleading interpretations of caseload stats?

Does Mcguire think these concerns about caseload stats still valuable?

Dwight

=============

http://www.fjc.gov/public/pdf.nsf/lookup/0020.pdf/$file/0020.pdf

How Caseload Statistics Deceive
Prepared by John Shapard, Federal Judicial Center
August 9, 1991
(NOTE: A draft of this paper dated May 2, 1991, contained an error in the parenthetical at
the end of the first paragraph on page 3: the word "divided" should have been "multiplied".
The only difference between this version and that of May 2 is correction of that error.)
1
How Caseload Statistics Deceive
Despite the various adages concerning statistics and lies, statistics don't lie.
Instead, we often mislead ourselves by misinterpreting statistics. Court caseload statistics
present numerous opportunities for this sort of self-deception. Obvious ways of looking at
caseload data and obvious nostrums about assessing a court's caseload are sometimes just
simply wrong. Their flaws are unappreciated not because they are hard to grasp, but
because we are conditioned to think about statistics using apples-and-oranges or dice-
throwing examples. Because significant time elapses over the life of many court cases, the
better statistical analogy is that of human populations. Failure to appreciate how the
lifespans of cases affect caseload statistics causes numerous misunderstandings. The
purpose of this paper is to illustrate three closely related misunderstandings about caseload
statistics, in the hope that a basic understanding of the problem can help prevent mistakes
on the part of the various parties charged under the Civil Justice Reform Act with trying to
improve the condition of court dockets .
Here is an example, to illustrate the problem. The standard index of case duration
in a district is the median time from filing to disposition for cases disposed of in the most
recent year. Suppose that the judges of a district , responding to increases in this median
time index, decide to improve the situation by working especially hard to clean up the
backlog of older pending cases. The judges begin working overtime trying cases that have
been awaiting trial, expediting or dismissing cases that have languished too long in the
pretrial process, and generally moving along or moving out all cases that they deem
overdue for some such movement. The effort and its results are impressive: annual case
dispositions increase, the number of cases pending decreases, and the median time from
filing to disposition goes way up! The key indicator of the court's "speed" indicates that it
has gotten slower than ever. The reason is not hard to see. Exactly as it intended, the court
disposed of a lot more old cases last year than it had in previous years. Because the cases
terminated last year include an unusually large number of old cases, but only the usual
number of young cases, the median age of terminated cases went up. The statistics are not
lying. We are deceiving ourselves in thinking that the median age of terminated cases is a
reliable indicator of average case duration.
1. Statistics based on terminated cases do not tell us about current caseloads.
The basic flaw in our thinking is this: terminated cases are not
representative of the court's caseload. The reason can be seen by considering the
analogy to human populations. In human populations as well as court caseloads, the life
expectancy of newborns or of newly filed cases is not necessarily the same as the average
age at death of persons who died last year or of cases disposed of last year. There is a
connection, but it is diffused, sometimes greatly, by the passage of time between birth and
death or filing and disposition.
Consider a district that has for many years enjoyed a very stable caseload: each year
2000 cases are filed, 2000 cases are terminated, and 2000 cases remain pending at the end
of the year. The median time from filing to disposition has long been 8 months. The
2
average1 time from filing to disposition has long been 12 months, and cases reaching trial
account for 10% of all cases terminated. Suddenly, in 1991, the case filing rate jumps to
3000 per year, the average age at termination drops to 10 months, and the percent of cases
reaching trial drops to 8%. It seems likely that the 1000 "new" case filings must have been
composed mainly of cases that are "faster" and "easier" than average. But that is wrong.
The truth is that nothing has changed except filing rate: the 3000 cases filed in 1991 will
average one year from filing to disposition, and 10% of them will reach trial. The average
age and trial rate statistics, which for many years told us the truth, are now lying.
The reason is not hard to understand. The 1000 additional case filings produce a
major increase in the number of young cases in the pending caseload (a "baby boom" of
sorts). Since the pending caseload is the supply of cases from which case terminations
arise, and since most cases are disposed of relatively quickly, the number of cases disposed
of at an early age increases dramatically. But there is no corresponding increase in the
supply of old cases, which arose when annual filings were just 2000 per year, so the
number of old case dispositions remains what it was in past years. Hence the average age
at termination drops. Similarly, because few young cases reach trial, the number of cases
disposed of after trial has not yet changed much. But the total number of case terminations
has increased due to the increased number of young-case dispositions, so the percentage of
cases disposed of after trial drops.
If our hypothetical court's filings rate either stayed at 3000 per year, or dropped
back to 2000 per year and stayed there, the statistical distortions would eventually
disappear. After a few years, the statistics would be back to normal, again showing the
historic one-year average age at termination and ten percent trial rate. But reality is not so
kind. Filing rates change, and in the long term trend they are often either increasing or
decreasing. When filing rates are continuously increasing, the median time from filing to
disposition will be constantly distorted downward, as will the trial rate, due to the constant
relative oversupply of young cases in the pending caseload. Conversely, decreasing filing
rates cause an upward distortion in both median age and trial rate.
2. How can you tell if a district is "staying abreast" of new case filings?
An oft-repeated nostrum is that to keep abreast of its caseload, a court must each
year dispose of as many cases as are filed Although that advice seems to make sense, the
unfortunate truth is that it is correct only under circumstances when it is too obvious to be
worth saying. If a court continues year after year to receive 2000 case filings and to
dispose of only 1800, there is obviously a problem. As can be seen from the example used
in the preceding section, an abrupt increase in case filings does not lead to a comparable
increase in case terminations, even when a court is staying fully abreast of its caseload in
the sense that it is maintaining a constant average age at termination. Conversely, when

1
Average is used here to represent the arithmetic average, or mean--the sum of the ages of terminated cases
divided by the number of cases. Annual reports from the Administrative Office of the U.S. Courts usually
report the median--half of all cases are terminated at an age that is at or below the median, and half at an age
that is at or above the median. The average age of terminated cases is usually about 50% greater than the
median.
3
filings are decreasing, saying abreast will yield annual case terminations that exceed annual
filings.
If the nostrum is false, how can you tell whether a court is "staying abreast?" The
answer is to track the ratio of pending cases to annual case terminations. If that ratio stays
constant, the court is staying abreast; if it decreases, the court is gaining ground--disposing
of cases faster--and if it increases, the court is falling behind. The ratio of pending cases to
annual case terminations is a good estimate of the true average duration (or life
expectancy) of a court's cases (the ratio gives average case duration in years; if multiplied
by 12 the result is average case duration in months).
It is useful to understand why the ratio of pending to terminated cases is a good
estimate of average case duration. The key point is that there is an absolute, albeit rough
arithmetic relationship between pending caseload and average case duration. To see that
relationship, consider a very simple example of a court that handles a single type of case,
each of which lasts exactly one year. Suppose the court receives exactly one case per
month, filed on the first of each month. This court must have exactly 12 cases pending at
any time (the case filed on the first of this month and those filed on the first of the
preceding 11 months). If instead each case lasts exactly six months, then the court will
have exactly six cases pending at any time. Although it is not intuitively obvious, the same
relationship exists--and can be mathematically proven--in respect to average case duration.
Provided that the mix of cases of varying durations remains constant and case filings are
continuous (i.e., they are not all filed in January, but are filed in roughly equal numbers
throughout the year), the pending caseload will equal average case duration (in years)
multiplied by annual case terminations. This point is key to the next and final topic.
3. The "momentum" of court caseloads.
Suppose a court that now has an average case duration of 24 months adopts a plan
for expediting case dispositions, with the goal of reducing average case duration to 12
months. What will this require? Consider the relationship explained in the previous
section. If average case duration is approximately equal to the ratio of pending cases to
annual case terminations, and if average case duration is 2 years, then the pending caseload
must include about twice as many cases as are annually terminated. To reduce average
duration to 1 year, the pending caseload must be cut in half. To accomplish that in the next
year, the court must dispose next year of twice as many cases as it did last year (provided
that annual filings do not change). To do it in two years requires that case terminations be
maintained for two years at a pace fifty per cent higher than current pace.
Are such accomplishments really possible? Probably not, although the answer
depends on how an increased pace of case terminations can be achieved. If it can be done
by methods that impose little additional demand on court resources, then it might be
possible to halve the pending caseload in a year or two. If instead the necessary methods
require a drastic increase in trials or other activities that place major demands on court
resources, then the pending caseload cannot be quickly cut in half without a major increase
in those resources.
Caseloads have momentum. The pending caseload is a heavy weight, and a court
can only be as fast as that weight will allow. To get faster, the court must shed weight.
4
Prescriptions and decisions about dieting will lead to disappointment if they are not based
on realistic goals and timetables.