I would like to draw your attention to a paper, see here, that uses the oral argument grades assigned by Justice Harry Blackmun to examine various issues relating to the effectiveness and influence of Supreme Court oral arguments. In the paper, which is forthcoming in Washington University Law Review, Tim Johnson, Jim Spriggs, and Paul Wahlbeck examine a random sample of 539 cases decided by the Court between 1970 and 1994 (spanning Justice Blackmun's tenure).
The aspect of the article that is most interesting, at least in my opinion, is the part of the paper that discusses those attributes of oral advocates that are most influential to the oral argument grades assigned by Justice Blackmun. Using such variables as whether the solicitor general is personally arguing the case, the amount of experience for the advocate arguing before the Court, and whether the advocate is an elite Washington attorney, former Supreme Court law clerk, or law professor, the authors run an ordinary least square (OLS) regression to determine which of the aforementioned factors most influence the oral argument grades assigned by Justice Blackmun. The results are not surprising as the personal appearance of the Solicitor General generally leads to a higher grade, and the presence of a former Supreme Court clerk or elite Washington attorney are also influential on the quality of an oral argument.
I do, however, question the grader in this case. Justice Blackmun appears to have been an especially harsh grader and the grades he assigned to several prominent legal figures are quite surprising. In the case of now-Chief Justice Roberts, who I believe is one of the great oral advocates of the past two decades, Justice Blackmun gave him only "average" grades in the twelve cases in the sample. In fact, in one of them, he said that Roberts did "better than usual," earning a grade of six on an eight point scale. When I clerked at the Court, I always made a point of being in the courtroom for Roberts' oral arguments as I always felt that there was much to be learned from his exceptional advocacy style and level of preparation. In addition, Blackmun gave only average grades to Justice Alito, and Justice Ginsburg did not fare particularly well in the sample either.
Note: Cross-posted at SCOTUSblog.com. Please also feel free to comment on my post on the challenges of data coding below, and I hope to have a post completed early next week on the use of descriptive statistics in empirical scholarship. I would like to thank Jason for permitting me to hang around for another week.
UPDATE: In response to my cross-post on SCOTUSblog.com yesterday, Howard Bashman wrote an "On Appeal" column for Law.com discussing the paper and explaining the importance of having experienced appellate attorneys arguing before the Court, see here.
Well (and if it's any consolation), from a research design standpoint one potential virtue of a single coder (Blackmun) is that the study side-steps intercoder reliability issues. That said, if Blackmun was uneven with his grades and/or applied inconsistent grading standards over time different (and important) measurement errors immediately arise.
Posted by: Michael Heise | April 06, 2007 at 01:51 PM