Of Interest from the MWPSA

Just returned from the Midwest Political Science Association conference and wanted to share some of "what I learned in Chicago."  Here goes.

First, we have discussed here issues of measurement, especially with respect to "the law" and "ideology."  A few presentations and comments of discussants are relevant to these discussions.  First, Sean Wilson's paper, entitled, "Modeling Justice Ideology Without Ecological Inference" argues that using career liberalism scores to predict votes is better than using the Segal and Cover scores in terms of model fit.  I suggested that using votes to predict votes is circular and the discussant (Brandon Bartels, soon-to-be assistant professor at SUNY StonyBrook) noted that use of the Martin-Quinn scores has been similarly criticized.  Martin and Quinn have a working paper on the question of when we can use their scores as independent variables that is interesting, though I remain unconvinced.  Seems to me that using percent liberal (or ideal points, which are based on votes) to predict votes shows us that judges are CONSISTENT, but not WHY they make the choices they do in the first place (which I find much more interesting).  Of course, this is all open to comment.  (And I do think Sean makes a good point that the attitudinal model operates differently in different issue areas.) Don Songer (South Carolina) brought up the issue of unanimous decisions during the discussion as well, and that's one area where the attitudinal model doesn't give us much.  We definitely need to better understand why the Court sometimes decides unanimously.  Thoughts?

Also interesting were a number of papers dealing with the influence of law on decision making, both on the Supreme Court and lower court levels.  Jason and I have already talked about our paper measuring the influence of legal interpretive strategies.  Bartels has an interesting paper that argues that the operation of preferences at the Supreme Court is influenced by law, using jurisprudential regimes to show that ideology has more room to influence outcome when, for example, rational basis review is appropriate than when strict scrutiny controls.   My coauthors (Jennifer Luse and Wendy Martinek) and I also use the notion of jurisprudential regimes to measure compliance by the Court of Appeals with the Supreme Court in our paper on Lemon, and Pam Corley (Vanderbilt) gave a very interesting paper that shows that TYPE of concurrence effects lower court compliance with the Supreme Court (and so, the Court has some control over its effect).   While I haven't read the papers or seen the presentations, papers by Cameron (Princeton) and Kornhauser (NYU), Jacobi and Tiller (Northwestern), and Lax (Columbia) look like they treat both law and the relationship between levels of courts in interesting ways.

Obviously, this barely scratches the surface in terms of what was presented at the conference and I encourage you to browse the MWPSA paper archive to read more!  (And would love for those of you who did attend, to use this opportunity to discuss the best work you saw presented.)

Comments

[Brandon Bartels]

I very much agree with Sara concerning the importance of recognizing the traditional tautological issue when employing measures of policy preferences.

However, while recognizing Sara’s concerns, I think a viable defense can be made for using various measures based on votes, including Martin-Quinn scores (which Martin and Quinn discuss in their working paper) and scores based on justices’ past behavior. I would be curious to hear what Sara and others think.

[By the way, at Midwest, I expressed reservations regarding Wilson’s use of scores that ranged over a justice’s entire career, favoring instead a “lesser of two evils” measure that used justices’ liberalism scores for terms previous to the one under examination. Alternatively, one could use percent liberal in the previous term.]

Consider first a preferences measure that uses the proportion of liberal votes cast by each justice in the previous term for an analysis at the choice level (that is, analyzing justices’ choices in a given set of cases over a given period of time). First, using proportion liberal for the PREVIOUS term reduces the tautology issue to a degree. Also, this vote-based measure would tap a justice’s average tendency to favor liberal or conservative policy. So it’s not necessarily a case of “using votes to explain votes.” Instead, the measure represents our best guess of a justice’s ideological leanings toward legal policy based his or her most recent tendency to favor liberal or conservative policy. Moreover, vote choice models require a preferences measure that can validly ascertain how much more liberal or conservative Justice A is compared to Justice B, and vote-based measures communicate these ideological differences between justices with greater validity than, e.g., Segal-Cover scores.

I think Martin and Quinn make some excellent points in their working paper (which Sara referenced) on whether their scores can be used as independent variables in vote choice models. Martin and Quinn essentially conclude that while the tautology issue is always a concern in a theoretical sense, practically speaking, under certain conditions, it does not represent a huge concern. [Of course, people should read the Martin-Quinn paper for further details] I also think it’s important to note that the tautology issue is one of a number concerns we should think about when employing preferences measures in vote choice models. Other important issues are (1) making valid comparisons about ideological differences between justices, which I talked about in the previous paragraph, (2) the issue of inter-justice comparability over time, and (3) accounting for the possibility that justices’ preferences may change over time. Martin-Quinn scores rank highly on all three of these additional criteria.

[Sean Wilson]

No one, I imagine, disputes that judging involves value choices and that law is not algebra. So if values display themselves in judging by definition, it is logical to collect these values as they manefest themselves in the work product through some objective accouting process. (As to the ultimate cause of the bias, let's put that aside for a moment).

Also, it must be remembered that it is not methodologically problematic to use career ratings in a logit model to explain discreet segments of voting data that, together, comprises the summary statistic. This is nothing other than an appeal to a "moving average." Of course, it may become problematic if the modeler simply uses a base rate to explain every set of votes analyzed, and each time calls the base rate "ideology." That is clearly tautological. But if one uses career ratings as a proxy for the extent of liberalism and conservatism possessed by justices, period, the model is not explaining votes with votes, but is merely making an empirically-based value assignment that is tested with moving-average logic.

If researchers don't like this, they can still use the moving average as a descriptive phenomenon in need of further explanation. That is, what causes the value assignments to move in the manner they do? Why are the values punctuated for some areas of voting (search and seizure) but not in others (core political speech)?

Hence, there is utility in using career numbers even if they are not endogenous.