Normative Implications of the Attitudinal Model

I am a social scientist.  Therefore, whenever anyone says the word "normative," I squirm.  I'll admit it.  But I have been thinking, lately, about some of the normative implications of widespread knowledge of the attitudinal model of judicial decision making.  The Law and Courts Listserv has been exchanging instances of hostility toward judges:  Justice Ginsburg's internet death threats for reliance on foreign sources of law; the movement to seize Justice Souter's property in New Hampshire; the new "J.A.I.L 4 Judges" Amendment in South Dakota.  Add to that list Pat Robertson's comments on This Week that activist judges are as bad as the 9/11 terrorists; the Senate Majority Leader's statements over the Shiavo case regarding Congress' power over the federal courts; Sensennbrenner's letter to the 7th Circuit on a sentencing decision; and on and on.  (The Brennan Center, of course, does an excellent job of tracking these threats.)  What is going on in terms of our confidence in our judiciary and over the presumed benefits of judicial independence? 

I think, just maybe, widespread appreciation of the fact that judges, like other politicians, are influenced by their policy preferences may be at least in part responsible for the erosion of confidence in the courts.  Is that possible?  As confirmation hearings become more partisan and politicians point to and decry ideological decision making by judges, and we, for our part, continue to lack appreciation of the role of the law in our research, the problem is exacerbated.  My first post discussed this question of how best to operationalize the law.  Maybe what's going on today makes the call for understanding even louder.

Allow me to offer one new piece of evidence that judges are not out there making exclusively ideologically-driven decisions, at least on the Court of Appeals.  A paper on which we're putting the finishing touches examines the variance in decision making on the Court of Appeals.  (Perhaps this fits into the category raised earlier on this blog about what we are learning through us of empricial methods.) Susan Haire, Erin Kaheny and I argue that it's not just the vote choice that is interesting about judicial (or other political actors') decision making, but also the consistency in that choice.   In other words, if critics are right, decisions should be driven by ideology AND they should be easily predicted on the basis of ideology.  What we find, though, is that there are systematic components in the error variance; that is, that the error term around the prediction of a liberal vote is not random but has systematic components including the complexity of the case, the size of the circuit, the dissent rate in the circuit, and the career status of the judge, among other things. (Paul Collins has a work in progress that examines the same question with respect to the Supreme Court with some similar results.)  What this means in the bigger picture, we think, is that concerns about purely political judging, at least at the Court of Appeals, is largely overstated.  (Not that we didn't already suspect/know this.)  Legal, case, and judge-specific characteristics influence the vote at the Court of Appeals level, as we know from much research on point.  But, legal, case, and judge characteristics ALSO matter in terms of the consistency of that ideological vote.  This should serve to give solace to those critics of the courts who think they are merely politicians in robes.  Would more evidence of this sort stop the attacks on the judiciary??

Comments

[Justin]

Quick question of clarification on Segal and Spaeth's version of the Attitudinal Model and the comment that their version controls for facts. I found that to be true in their book where they discuss search and seizure, but I did not think and have not found that they did that for the economics and civil liberties cases in the two articles. Did they control for facts when running regressions in those cases? If so, can you give me a citation on where that would be?

[Christopher Zorn]

I like this discussion, in part because it gets to some of the issues that Sara raised in an earlier post: how one might/can/should go about incorporating/operationalizing law, specifically in the context of the Supreme Court (where, I'm sure, that word means something somewhat different than it does elsewhere).

Two points: On one hand, I buy the "attitudinal model," at least in its more carefully-delineated forms. Ideology (in the Rokkanian sense of a more-or-less consistent set of primitive beliefs ordering affective priorities) affects Supreme Court decision making, fine. *When* it does so, and to what extent, and if/why such variation exists, we are only beginning to explore (and my hunch is that "law," broadly defined, has at least something to do with it). The bigger point that people seem to be fixated on is *how much* it matters; and that, as with so many things, depends a lot on your perspective.

If I use Jeff S.'s original search-and-seizure data, and I estimate a model that has all the case-fact variables plus one for Segal-Cover scores on the right-hand side, my trusty software tells me that I can explain about 18 percent of the variation in the votes with such a model. If I then take the same data, and average the 14 justices' voting scores over the whole 1962-1985 period, and regress *that* on their Segal-Cover measure alone, my R^2 is closer to 0.45. The point being: Aggregate voting scores are inherently more explicable than are individual votes; we will *always* do a better job of predicting ("explaining"?) the former than the latter. So, yes: Some care needs to be taken to ensure that we're clear about what we're explaining.

But, to my mind, all this is really just preface to the real game. If our interest is an an "attitudinal" one -- that is, we want to know about the extent to which ideological/political preferences influence judicial decision making -- then law is, at best, a nuisance: It stumbles into cases, knocking over tables, drinking all the wine, confounding itself with everything else, and generally preventing us from getting at the attitudes --> decisions link in a nice, clean way. In that case, our best bet is to attempt to "sweep out" law's influence, to get at the marginal impact of ideology.

On the other hand, if (as I suspect is generally the case) we really want to understand holistically what's going on in judicial decisions, then we need to get to theorizing. "The Attitudinal Model" is all well and good, as far as it goes, but it needs to be a leg up on a broader understanding of what judges and justices are about, rather than just a subject of criticism. (Newton's "shoulders of giants," and all that). That means paying attention to legal theory (if only instrumentally, because we know judges do), and it means knowing more about the history, psychology, and politics of what justices do.

[Sean Wilson]

You know, Howard, the problem is that there are attitudinal CLAIMS, and there is statistical methodology. What you are saying is that the true "attitudinal model" envisions a context-dependent decision environment that necessarily relies upon jurisprudential structures and factors. But that is NOT what they argue they have proved. They deny a Dworkin effect (in writings); they assert judges suffer from motivated reasoning to the extent that they deceive themselves when talking about the gravity of law; they tell others who believe in law as a structural force upon the mind that they are believing in a Pollyanna fairy tale; they say that 60-70% of justice choices are empirically accounted for by political values; and that Bush v. Gore exemplifies what they are talking about.

The question comes down to this: (1) what do they say (now) about Dworkin?; (2) Do they believe they are working within a hard-core realist framework (skepticism) or an objectivist one (Holmes, Lewellyn); (3) Do they see a difference between hard and soft politics?; (4) Do they deny that legal orthodoxy structures (and thereby affects) decision making; and (5) Do they deny a Kantian or Socratic influence upon the mind?

If you tell me that the true attitudinal model supports these things, I would say in response that there is no attitudinal model, there is only attitudinal rhetoric. To me, I would rather keep it straight forward: there are attitudinal claims that are exaggerated and not supported by the researchers' creation. In fact, the statistical data they created supports much of what philosophic and legal culture had been saying for years (which is not bad, of course -- having empiricism validate good theory is always welcome).

[Howard Gillman]

Thanks to Chris, but I don't see the problem with my characterization. Chris verifies that the dependent variable in the SCAM/R model is "variance across different justices' aggregate voting patterns" rather than votes in individual cases. Thus, the model is not really trying to explain the justices' voting behavior; it's trying to explain variation in the aggregated/summary patterns. To put it another way: while Jeff and Harold famously wrote that "Rehnquist votes the way he does because he is extremely conservative [and] Marshall voted the way he did because he was extremely liberal," it would be more accurate to say that their model shows that "Rehnquist has a less liberal overall voting record than Marshall because he is more conservative than Marshall."

And so maybe we just disagree on the "metaphysical significance" of the point. For me, it's a matter of being careful about what the evidence in SCAMR actually demonstrates. If people get the (mistaken) impression that the Attitudinal Model seeks to explain the justices' votes on the merits then they will make a huge mistake in assessing the model's explanatory power on that dependent variable. To get back to Sara's initial comment, that means that most of what is driving votes in individual cases is NOT conventional political preferences -- and perhaps that's an important lesson to emphasize (even though I don't think it's the lesson that most people associated with the discussion in SCAMR).

Finally (and this is consistent with something Sara said earlier), it's my understanding from earlier exchanges with Jeff that, as applied to individual votes, the attitudinal model (properly specified) REQUIRES the incorporation of "case facts" as an element of the analysis, and that it is possible -- even likely -- that the most relevant "case facts" for completing the model are JURISPRUDENTIAL factors.

[Sean Wilson]

Chris:

Indeed, the problem comes with the introduction of ecological inference into the model.

Aggregation pads the product, plain and simple. That's what my forthcoming work says.

[Christopher Zorn]

Howard said:

"Importantly, the model in SCAMR is NOT explaining votes; it's explaining the variance within each justices' aggregate voting patterns."

Actually, that's not quite right. By aggregating across cases for each justice, the resulting model is in fact explaining the variance *across* different justices aggregate voting patterns; the variation *within* each justice's voting behavior is effectively averaged away (which is why, among other things, the fact/assertion that justices' preferences and behavior remains constant over the course of their careers is so important to this work).

Moreover, the fact that aggregate (justice-level) ideology measures better explain aggregate (justice-level) voting statistics than they do case- or vote-level outcomes has no metaphysical significance at all; it's merely due to aggregation effects. As it happens, Mike Giles and I had a paper on this issue in the Law & Courts newsletter back in Spring 2000, and I have a related paper forthcoming in Political Research Quarterly that deals with a very similar issue (that is, what is the proper "unit" on which to group vote-level data).

[Howard Gillman]

To answer Michael Heise (and in slight disagreement with Sara's claim that "the Attitudinal Model is that it predicts votes using ideology in light of the facts of the case") -- when SCAMR reports (p.322) that Segal/Cover data "explain 60% of the variance in justices' decisions," the dependent variable is NOT tens of thousands of individual votes; it's the statistical summaries of the justices overall tendency to vote liberal or conservative. In other words, the variance being explained is why Scalia is at 29.6% versus Blackmun at 52.8%. Importantly, the model in SCAMR is NOT explaining votes; it's explaining the variance within each justices' aggregate voting patterns. To put it another way, Segal and Spaeth are not correct when they say on p.322 that the dependent variable are the VOTES of all justices ... in all formally decided civil liberties cases." (Figure 8.2 makes it clear that the dependant variable is the overall liberalism score.) When one tries to use Segal/Cover to explain votes themselves the results are dramatically lower (1/3 of the explanatory power versus explaining the variation).

I think it would be very useful if more scholars working in this tradition appreciated these differences. Among other things it would be more difficult to say that the model proves that SC decision making is overwhelming influenced by conventional political ideology. In fact, it is overwhelming influenced by other factors.

[Sean Wilson]

Hello Sara!

The problem with putting all the attitudinal eggs into the case facts basket is this: legal and philosophic culture never denied that ideology was tempered by case particulars. Indeed, are you guys now agreeable with Dworkin? Justices engage in a cognitive process that involves facts, prior choices, and political values. Some recipes have more spice (Scalia). Others rely more upon problem solving (Bryer, Harlan).

We will be a happier empirical universe once we our all on the same page.

[Sean Wilson]

William: I do not doubt that choosing centrist policies results in a distribution of power in society that would be different but-for the selection of centrist outcomes, and therefore is "political" in the constitutive sense of the world.

To that end, I do not doubt that judging is "political." (Under this definition, virtually everything is political -- including science).

What I advocate is linguistic clarity. If we mean to offer a critique of judging that is not falsifiable -- voting left is political, voting right is political, voting centrist is political (presumably non-voting is political) -- then we should at least warn our audience that much of what we have found in our conclusions is inherent in the way we are talking.

If, however, we are using the term "ideology" to mean a left/right "bias" (as in affiliation with a team), we have to say only what can be said: directionality is not as sexy on the Court as Segal and Spaeth would have you believe.

The reason for this is that Segal and Spaeth use the term "ideology" in a way that is a language game. Sometimes the word means direction; other times it means "volition" (making a choice). They don't measure each phenomenon separately. They offer one big package where the concept means one thing in one paragraph, then changes in the next.

This game of linguistic dodge ball allows the attitudinal model to never be very wrong.

Therefore, I do not deny that centrist judging is political; I only deny that it is non-directional, and that directionality on the Court is lower than you think and on the decline.

[William Ford]

Here is a tiny bit of evidence in favor of Sara’s point that the public recognizes the role of judges’ policy or political views, even without reading the work of political scientists: In a Gallup poll conducted in June 1946 -- two year’s before the publication of Pritchett’s Roosevelt Court -- 40% of the public agreed that “the Supreme Court decides many questions largely on the basis of politics.” Another 22% didn’t know. The remaining 38% disagreed. (Gallup Poll #373.) Then again, Pritchett did publish a relevant piece in the APSR in 1941 ....

[Sara Benesh]

That's a good point. I suppose the controversy surrounding the use of foreign law is less germane to my question than the other controversies we've seen surrounding courts lately (Schiavo case, Robertson's comments, Sensennbrenner's letter, etc.).

[William Ford]

Is there something odd about attributing the unhappiness over the Court’s recent citations of foreign sources to the attitudinal model? The people who are most upset about these citations take them very seriously, as if they actually mattered to the Court’s decisions.

If you view these foreign sources as serving decorative purposes only -- at least in these recent cases -- then there is less reason to worry about them. The problem with these decisions, if there is one, is to be found elsewhere. Instead, people talk as if the foreign cases actually compelled the outcomes. In other words, they talk the legal model talk, not the attitudinal model talk.

[Sara Benesh]

My understanding of the Attitudinal Model is that it predicts votes using ideology in light of the facts of the case. Therefore, it is a model used to predict/explain votes NOT the variation around those votes. Sean's version of the Attitudinal Model does not include controls for the facts of the case and so does not explain as much as the Attitudinal Model as delineated by Segal and Spaeth does. One can presumably posit that ideology causes votes, but the AM posits that ideology *in light of the facts of the case* causes votes. There is a difference, obviously.

As to Professor Henderson's comment, I know no one reads our work. My contention is not that people know what the "Attitudinal Model" is per se, but rather that the understanding that politics influence judicial decision making has trickled down to public consciousness throughout the years and that that knowledge, along with the way judges are nominated and confirmed, might serve to influence the public's perception.

[William Henderson]

Sean,

What is a "pragmatist tendency"? Voting a way that seems centrist and thus defies simple modeling? Lots of people have labeled O'Connor as a pragmatist--usually in a very flattering way. But it could be argued that pragmatism is evidence of a highly politicized judiciary because it reflects a jurisprudence that picks the most politically appealing outcome and reasons backwards. In a recent HLR article, Posner essentially says that virtually any legal outcome can be propped up with citations to existing case law. Posner is also referred to a pragmatist.

It may be very hard to model how the behavior of judges changes when the public somehow expects them to be political. That dynamic might describe our current time period, especially at the Sup Ct level.

Re Sara's initial point that the attitudinal model may be responsible for changing public attitudes toward the courts, I doubt it. Most law student have no idea that this literature exists. And many law professors are not familiar with it either. See Rosenberg, The Great Divide.

[Sean Wilson]

The plain and simple fact is that justice ideology at the level it is observed by Harold Spaeth and Jeff Segal only accounts for about 15-30% of the overall voting variance on the Court, depending on whether you use Segal Cover scores as your values or career liberal rating, and depending on whether you model the entire docket or only civil liberties cases. Anyway you slice it, Segal and Spaeth's own data tell us that 70 - 85% of the justice votes are due to factors OTHER than ideology at the level the trait is said to be observed by the researchers. I have a paper that I am updating that will soon be out there.

Gone are the days when political scientists can say that ideology at the level we actually observe the phenomenon explains 60%or better of the Court's votes.

Of course, we must be careful of the language game. Ideology in this sense simply means propensity for direction; it doesn't mean a world view absent such a propensity.

The reason why ideology is not so sexy when researchers model it properly is because of two simple realities: (1) there are too many non-directional voting patterns in the data; and (2) the scale of justice values is squishing. If Rehnquist started out in the single digits (say, 5% liberal) and the most liberal justice was in the 90%, and you had very few centrists, you would have an amazing model. Instead, you have an iffy one because of scale squishing and pragmatic tendencies. The problem is getting worse as time goes by (my Chicago paper shows this).

In essence, the attitudinal model's own data shows us that ideology is becoming less a factor, not more. The age of pragmatism cometh.

[Michael Heise]

I also would appreciate confirmation of Howard's understanding. I seem to recall some animated listserv discussion a while ago on this point -- that is, whether the SCAMR data speak to variance rather than actual votes. The difference, of course, is obviously important.

[Howard Gillman]

Interesting question, Sara. In addition to exploring the relevance of more data along the lines you described I wonder if we should also consider the sort of language used by Harold and Jeff in The Supreme Court and the Attitudinal Model Revisited. There are places where it is suggested that only those who are naïve, ignorant, or hopeless romantics would continue to believe that justices are something other than unrestrained (and sometimes even shameless) promoters of their personal policy preferences.

Whether or not that language is incautious may depend on some clarification on exactly what Jeff and Harold argue in SCAMR. It is widely assumed that Harold and Jeff argue that ideology overwhelmingly drives the Court's decision making. However, despite some misleading language in SCAMR, it is my understanding (and please correct me if I'm wrong) that the data is only designed to explain variance in votes, not explain votes. If this was better understood then does that also have relevance for your question?