Are ordinary law professors as well-equipped as PhD social scientists to carry out content analysis of judicial opinions? There are reasons to think that they might not be. Content analysis, after all, is a standard technique used in communications, political science, sociology, and other disciplines. It is a common part of the methodological training of social scientists. Legal scholars, meanwhile, often “discover” the techniques of content analysis during research, intuiting many of its essential components on the fly.
On the other hand, there are reasons to think that non-PhD legal scholars could do content analysis reasonably well, even without formal training. In some ways, the method resembles the traditional lawyerly job of reading opinions and trying to evaluate them by using consistent criteria.
So which is it? Does PhD training matter in the execution of content analysis?
- First, a word about trends over time. About two-thirds of the 125 studies we located from the past half-century include a PhD author. Surprisingly, this ratio has not increased in recent years, despite changes in the composition of law school faculties.
- Second, the presence of PhD authors does make a difference in citations to methodological literature. Among the PhD-authored studies, three quarters cited to methodology literature, compared to half of the non-PhD-authored studies.
- Third (and this surprised us), the presence of PhD authors did not make much difference in efforts to demonstrate coder reliability. While it was predictable that 71% of the non-PhD studies included no discussion of coder reliability (such as measuring consistency among coders), fully 62% of the PhD studies also included no discussion of coder reliability. The non-PhD studies used student coders 40% of the time, while the PhD studies used student coders 34% of the time.
- Finally, the studies by PhD authors did tend to use statistical techniques more often to analyze the coded results. About two-thirds of the PhD studies used multiple regression analysis, compared to 40% of the non-PhD studies.
How would you expect PhD-authored studies to differ from those conducted by authors with legal training only?