Ward Farnsworth (Boston University Law) has posted The Use and Limits of Martin-Quinn Scores to Assess Supreme Court Justices, with Special Attention to the Problem of Ideological Drift. The Abstract:
This paper explains and examines the use of Martin-Quinn scores to assess the behavior of Supreme Court Justices. It is a reply to a recent paper by Lee Epstein, Jeffrey Segal, Andrew Martin, and Kevin Quinn which claims that the policy preferences of most Justices change during their careers; the authors of that paper suggest that this should cause Presidents to reconsider the use of nominations to try to change the direction of the Court. The authors base their findings on changes in the Justices' Martin-Quinn scores, but the meaning of those scores has not yet been fully explained in plain English. The present article attempts such an explanation. It discusses the features of judicial behavior that the Martin-Quinn method accounts for and does not account for, the limitations of the method, and some questions about the method that remain to be answered.
The limits of Martin-Quinn scores raise some doubts about the authors' conclusions. Martin-Quinn scores are generated by simply observing patterns of coalition voting among the justices without paying any attention to what the cases are about. The authors assume that all voting is ideological, so any change in the patterns of the coalitions the Justices form is taken to show changes in the Justices' ideologies. There are various reasons to question this chain of reasoning. The most important is that the authors' model treats all cases as equally important and revealing. So if a Justice starts to vote a little to the left of where he formerly did (relative to his colleagues) in any area of law, this may cause a change in how the Martin-Quinn model views his entire ideology - even if his voting has been consistent in most areas of great public interest. So when the authors find statistical changes in the behavior of Justices, those changes may not (and in some cases do not appear in fact) to amount to shifts that would have mattered to the Presidents who appointed those Justices in the first place.
Further, the authors write as though predictability were monolithic: either the behavior of Justices can be predicted or it can't be. They don't take into account the possibility that risks of ideological drift are greater among some nominees than others, and that Presidents and others can foresee this. Nominees who have done extensive service in the political branches of a party (such as Rehnquist, Scalia, Thomas, Roberts, and Alito) are more reliable bets than nominees without such political experience (such as Stevens, Souter, and Kennedy). Nominees of the latter kind often are chosen by Presidents precisely to avoid tough confirmation fights; the risk that they will drift ideologically is perceived by everyone from the start, and is the reason why such nominees are not opposed as vigorously. Most cases where Justices have changed in ways that would have disappointed their nominators appear to involve nominees who were understood to be in the relatively risky group from the start. So while the authors' findings are interesting, they don't yet seem to call for much revision in the thinking of those who choose Supreme Court nominees or argue about them.