Both the existence and content of this blog show that empirical legal studies has made extraordinary progress in recent years in the field of applying empirical analysis to the effects of the law. However, the progress of empirical legal studies in analyzing legal institutions themselves -- courts, judges, and cases -- is more haphazard.
The central limitation is that the primary test of judicial ideology is inadequate. The traditional approach is to count the number of case outcomes categorized as liberal and contrast them to the number of cases labeled conservative. But case facts and the questions of law they present are not dichotomous; they represent the full breadth of legal inquiry. One case that is labeled liberal may be far more liberal than another case with the same label. Michael Bailey and Kelly Chang equated this mode of analysis to giving students a calculus exam and an addition quiz, and treating a score of 80% on each as equivalent.
In recent years, scholars have turned to proxies and indirect measures to capture judicial ideology. Notably, Michael Giles, Virginia Hettinger and Todd Peppers have pursued the use of the proxy of the ideological score of the appointing president to substitute for a judicial score. Another indirect measure is to use media reporting, using content analysis of newspaper editorials, as Segal and Cover have done.
One of the most exciting developments that provides a direct score of judicial ideology is that developed by Andrew Martin and Kevin Quinn. Analogous to Keith Poole and Howard Rosenthal’s measures
of congressional ideology using rollcall votes, Martin and Quinn have
developed a rank ordering of Supreme Court justices on a constant
standard, using the composition of coalitions to develop their scores.
This allows for historical comparisons of justices who served on
different courts, as well as tracking the movement of individual
justices over time.
My colleague Vanessa Baird and
I have used Martin Quinn's measure of judicial ideology to develop a
new measure of the ideological positions of case outcomes. Explored formally and in application,
we propose that the mean of the majority coalition is a more nuanced
and generally superior measure to the simple traditional test of the
labels liberal and conservative.
These measures offer a promising way of continuing the progress in empirical legal studies generally in measuring judicial ideology and the ideological placement of case outcomes.