The journal that I edit, The
Justice System Journal, is a peer reviewed public law and judicial politics
journal that publishes articles on courts and court administration, very
broadly defined. The journal is available online through Westlaw and Hein on
Line. Unlike The Journal of Empirical Legal Studies, we operate on an exclusive
submission basis. We serve two audiences.
First we have an audience of court administrators and court professionals.
These readers want information both practical and theoretical about the state
of courts and court administration as well suggestions as to how to improve
performance. Our second audience, and the one most of you would be interested
in, is that of social scientists and those in the legal profession.We seek quality social science work, and will
accept different methodological approaches.
The practice of some previous
editors was to insist that submissions to the Journal forgo most statistical
and methodological language that would, in their opinion, be incomprehensible
to much of the audience of court administrators. While I understand the
concern, that begs the question of where one draws the appropriate line in
talking to our various audiences. The prior editor did not allow any use of
numbers, data or the use of statistical of methodological language and
analyses. Thus, authors could not discuss actual data or methods and could not
present coefficients, only asterisks to represent statistical probabilities of
occurrence.There was a thought that
academic authors could save their detailed data driven work for a second
version and publish that in other journals. This “secondary market” of
publication rarely, and I think appropriately rarely, occurred. Once submitted
to us, authors were understandably reluctant to submit essentially the same
piece to another journal.
I am not criticizing past practices
for this, but I am trying to find the right mix. I will include methodological
analyses and full reporting of data and results. I do ask, however, for greater
clarity and greater explanation of data and analysis with the understanding of
our dual audience. I am now trying to decide about formal theory analysis. Can
they be explained so that a diverse audience can appreciate that?
Since this is a blog of law professors and political scientists and connected to the Journal of Empirical Legal Studies, a journal that uses peer review but also allows multiple submissions, an issue I want to raise is publishing venue. The ability to speak to more than one audience is a tremendous benefit to political scientists who research and write in judicial politics and public law. It is very satisfying to see your work cited and discussed by those outside the discipline and this helps in establishing a national reputation, always a critical part of tenure and promotion decisions.
However, for those of us in the social sciences there are some pitfalls. You have to be careful to consider the weight of various publication outlets, make sure your department will give you credit for those publications, and consider whether potential employers want publications in journals outside of political science. One thing I have noticed in participating in job searches over the past several years is the increase in the number of law review articles published by candidates for judicial politics and public law positions.However, since these are not peer reviewed, how much weight should a hiring committee give to these types of publications?It is often difficult for hiring committees to evaluate law review articles. To most of my colleagues it is difficult to assess the article and compare it to a work that has gone through peer review.
Political scientists are familiar with the hierarchy of political science journals and can infer information and potential quality of the article from the journal in which a peer reviewed article is published.Most departments usually have one or at most two faculty specializing in public law and judicial politics.Because of this, most departmental search committee members have only a vague idea of the submission and publication process for law reviews, or how to assess their worth. Many of my non judicial departmental colleagues think of them as the equivalent of other non peer reviewed publications such as book reviews and book chapters. I think this can be unfair because many times law review articles have gone through an informal peer review process through presentation at symposia and because they are often available online prior to publication in the law review and thus accessible for comments and suggestions. Because of this attitude my recommendation is to concentrate on peer reviewed submissions when building a record for a first job out of graduate school or for a job in another political science department while untenured. After tenure I am not so sure, but I find myself still submitting to peer reviewed publications, and only thinking of law reviews for articles that have methodological problems, such as a small sample size.
Thanks Jeff for the comment to my first post and let me amplify in this second post. I agree that accurately measuring judicial ideology is crucial to so much of our research in judicial politics. To answer Jeff’s concerns, the scores are very highly correlated with the Giles et al. common space scores (pearson=.768), and the Giles’ scores do not predict lifetime liberalism of federal judges significantly better or worse than the Nixon/Howard scores (pearson correlation=-.278, p<.01). Having said that, however, there is a problem with the Giles et al measure in predicting judicial voting behavior. When there is no home state senator of the same party as the president, Giles et al. scores the nominee at the same ideology as the president. But because the point estimates for presidential common space scores are relatively extreme (see Clinton et al. 2004), this assumption has the effect of making these judges ideologically extreme. Indeed if both Indiana senators were Republican then Hamilton would have the same ideology score of Obama, which although not as liberal as conservative commentators make it out to be, it is considerably to the left of Bayh and our measure.
The assumption is a significant one, because senatorial courtesy was absent in about one of four of 395 appointments to the U.S. Circuit Courts of Appeals. Scott Graves and I had to confront this problem in our book on recess appointees and comparing pre and post recess appointment voting. A recess appointment by definition has no home state senator and the Giles et al. measures just did not make sense and that the voting behavior of judges does not support the extreme. Judges appointed by Republican presidents exhibit only a slightly more conservative voting record when their appointment was unconstrained by senatorial courtesy, and the difference is not statistically significant by a 1-tailed test. Judges appointed by Democratic presidents exhibit only a slightly more liberal voting record when their appointment was unconstrained by senatorial courtesy, and the difference is not statistically significant. In comparison, the Nixon/Howard ideology scores show that the pattern much more closely matches observed voting behavior of judges. The scores exhibit slight extremity in the president’s direction, when the president’s nomination is unconstrained by senatorial courtesy, and the difference is only marginally significant.
Of course one could argue that in the absence of a moderate Democratic senator like Bayh, Obama would have appointed a more liberal judge, but given the pool of available judges that does not seem likely and the above referenced data do not support such a conclusion
I'd like to take this opportunity to introduce and welcome Bob Howard, who will be Guest Blogging with us this week. Bob is currently Associate Professor of Political Science at Georgia State University and Editor-in-Chief of the Justice System Journal (published by the National Center for State Courts). Bob holds a JD from Union College, a MA from Suffolk University Law School, and a PhD from SUNY Stony Brook. His research often involves specialized courts and tax policy. He's also written two papers with Jeff Segal expressly attempting to operationalize "law" and test for its influence (exploring the effect of originalism and judicial role perceptions, respectively, on Supreme Court decision making). Hence, he is the perfect person to talk about issues in empirical legal research and its publication, and we are anxious to discuss the issues he raises. Bob, welcome!
I would like to thank the editors for this opportunity to blog on the ELS site. Among other titles and roles, I am the editor of the Justice System Journal and I will blog more about that in another entry, but for now I would like to blog about the recent nomination of David Hamilton to the U.S. Court of Appeals. I was asked to blog about this for Rowman & Littlefield, soon to publish a book on judicial recess appointments by Scott Graves and yours truly, and I thought my views on the nomination would be of interest to ELS readers.
Hamilton was widely viewed as a moderate and strongly endorsed by his home state senators, both Democrat Evan Bayh and Republican Richard Lugar, although several conservative websites decried his nomination and viewed Hamilton as an ideologically extreme liberal judge. Hamilton had his first hearing in front of the judiciary committee on April 1.
Of course the question that I want to address here is how moderate or liberal is David Hamilton? Is he an extreme ideologue as his detractors contend or the moderate and sober jurist as portrayed by his defenders? How do we know? There is no generally accepted measure of lower court ideology. A judge’s own partisan affiliation and the ideology of a judge’s appointing president have often been employed as useful surrogates of judicial attitudes. Scholars have sometimes ignored the ideology of the judge, and inferred his or her ideology from that of the appointing president. For example, Tate and Handberg (1991) proposed a measure of the ideology of the appointing president: -1 for ideologically conservative and presidents, 0 for nonideological presidents, and 1 for ideologically liberal presidents.
A recent and now widely used ideology measure was devised by Giles, Hettinger and Pepper (2001). This uses the Poole Nominate scores of the home state senators or of the nominating president if there is no home state senator of the same party as the president. The use of nominate scores allows direct comparisons with other political actors such as Senators, Representatives and Presidents, but leaves a very narrow ideology range for nominees, and all the nominees of a President who do not have a home state senator of the president’s party will have the same score.
To remedy these problems, a former colleague of mine, Dave Nixon, developed a more direct measure for each judge. We first used this in a paper published in the Washington University Journal of Law and Policy (2003), and Scott Graves and I use it in our forthcoming book and I use it some other publications and it will be in another forthcoming book of mine of Courts and Tax policy. The calculation begins by using the nominate scores of congressional representatives who later served as federal judges as a formula for determining a nominate score comparable to Nominate scores for other federal officials. The formula then uses various circumstances surrounding the appointment such as unified government, wartime, party of the judge and party of the president, among other factors. Unlike the Giles, et. al. scores the Nixon/Howard scores allow for differences for judges even if appointed from the same state by the same president. The scores range from about -5, most liberal to + 5, most conservative.
Given all that, what is Hamilton’s ideology? The Giles, et al scores place him at the exact same score of Evan Bayh, his home state senator, which is -.223, liberal to be sure, but moderately liberal, in fact just about smack dab in the middle range of potential Democratic appointee scores. The Nixon/Howard measure, on the other hand gives Hamilton a score of -.151, again liberal, but considerably closer to moderate than the Giles score. By either measure Hamilton is far from extreme. By our measure he is about as moderate a Democratic appointee as Republicans and Conservatives can legitimately hope for from a Democratic administration. Given the vehemence of the opposition, it does not bode well for any sense of post partisanship in forthcoming judicial nominations.
Last month I noted a paper (by Bob Lawless (IL), Angie Littwin (Texas), Katie Porter (Iowa), John Pottow (Michigan), Debb Thorne (Ohio U.), and Elizabeth Warren (Harvard)) that empirically assesses the recently reformed bankruptcy code. Given all the scholarly activity it ignited, an update is warranted.
The original Lawless et al. paper stimulated a critique from Rafael Pardo (Seattle). Lawless et al. respond to Pardo's critique (at: 83 Amer. Bankruptcy L.J. 47-61). Finally, to push the debate further, Pardo responds to the Lawless et al. response here.
As previously promised (here), over at PrawfsBlawg you'll find John Pfaff's (Fordham) second post in his promised series on ELS. In it he overviews some of the major technological advancements that have fueled ELS. Well worth a read.
Owing to my co-ELS bloggers' modesty, it falls to me to highlight a recent paper by Henderson (IU-Bloomington), Zorn (Penn St.), and Czarnezki (Vermont). In the spirit and growing tradition that distinguish empirical work, their paper, Working Class Judges, builds on prior work (by Scott Baker) and leverages the dataset that Baker makes publicly available (an emerging norm that fuels empirical work).
In a prior paper, Baker concludes that "higher judicial salaries would have virtually no effect on the performance of federal appellate judges." After reanalyzing Baker's data, Henderson, Zorn, and Czarnezki "qualify Baker's interpretation of his results, at least with regard to judges located in the 'Top Five' legal markets of New York, Chicago, Los Angeles, San Francisco, and Washington, D.C.
In his original analysis, Baker relies upon the average law firm partnership compensation, adjusted for years in practice and region, to estimate the forgone income - and hence opportunity costs - of each federal judge. Baker explicitly anticipated the possibility that this variable would understate the opportunity cost in large legal markets; thus, he included a Top Five variable plus an interaction term, which captures the effect of forgone earnings when a judge is located in one of the nation's five largest legal markets. Baker's discussion, however, does not formally address the significance of the interaction term, which requires some additional steps to properly interpret.
Based on our reanalysis of Baker's specifications, it appears that judges in the largest legal markets often behave differently than their smaller market counterparts. Specifically, the lower judicial salaries in Top Five markets strongly correlate with behavior Baker characterizes as "ideological" or "influence-motivated." Conversely, while lower judicial salaries in small markets correlate with longer delays in issuing opinions, the exact opposite effect describes the behavior of judges in Top Five metropolitan areas."
Over at Prawfsblog John Pfaff (Fordham) begins a series of posts on challenges confronting (or will soon confront) empirical legal scholars. John touches on a number of important issues that he promises to get back to in greater detail in subsequent posts. Well worth reading, and I look forward to more.