We are not proselytizers of content analysis. It has certain advantages in legal studies, but also substantial limitations, compared with conventional legal analysis. A full exegesis would take a dissertation (at least), but here are a few summary thoughts.
We think the most successful applications have been those that:
- Debunk conventional wisdom by showing that widely-held beliefs are descriptively wrong about what’s actually in the caselaw.
- Describe the landscape of a body of caselaw, much like a naturalist might explore a new continent or even a familiar patch of woods, by turning over stones and carefully documenting what crawls out.
- Use regression analysis to uncover hidden patterns in complex fact-specific areas of law
- Study particular techniques of opinion-writing, such as syntax, semantics, citations, or reasoning style.
These applications put content analysis to its best use in legal studies because they highlight the empirical weaknesses of traditional legal methods, and they focus on aspects of opinions that are objective and primarily descriptive.
Less successful, or potentially misapplied, uses are those that try to:
- Predict the outcome of future cases as practicing lawyers might do
- Explain definitively what legal or factual elements cause courts to rule one way or the other
- Bring the appearance of scientific rigor to a normative critique of the law (e.g., to show that courts got it wrong/right or made bad/good law)
The primary problem in the first two on this “bad” list is the inherent bias and circularity (or “endogeneity”, as social scientists would say) in how factual elements are reported in opinions. Our skepticism is not to disbelieve judges when they give us the reasons for their decision. Instead, we question whether factual and legal elements can be cleanly separated looking only at an appellate decision. Also, we doubt that the facts reported are reliable guides to real-world facts because of how they are filtered and distorted by the litigation process.
Do you agree that the first batch of purposes is each a legitimate use of content analysis? Would you like to defend any of the second batch of questionable uses?