Empirical Legal Studies

The Federal Circuit Dataset project (v.3)

While Jason Rantanen's (Iowa) third update to The Federal Circuit Dataset Project will certainly interest empirical IP scholars, the Project's broader implications speak to empirical law scholars in all fields.

The third edition advances the existing Compendium of Federal Circuit Decisions in at least two main ways. First, the dataset now includes document numbers for all archived documents. Second, the dataset now facilitates increased understanding about what happens to appeals, including information about appeal pendency and origin.

Setting aside the dataset's utility to empirical IP scholars, the Project itself reflects a growing commitment to the hard and often under-rewarded work of dataset creation as well as an impulse to make such data (and related codebooks, etc.) publicly-available. Such impulses need to be rewarded, promoted, and nurtured. The paper's abstract follows.

"This paper introduces the third generation of the Federal Circuit Dataset Project. The third generation builds on the existing Compendium of Federal Circuit Decisions with the following enhancements:

• The addition of a dataset of all dockets for the Federal Circuit since 2000. These dockets are linked to the documents in the Compendium through the Appeal_Dockets field in both datasets.
• New fields containing additional information about Federal Circuit decisions.
• More complete data in existing fields that require manual coding.
• Parallel archiving of the document and docket datasets on the Dataverse.

In addition to describing the third generation of the Federal Circuit Dataset Project, this paper also provides updated descriptive statistics about Federal Circuit decisions and new statistics on appeal dockets."

Technology and Legal Citation "Stickiness"

An interesting citation paper caught my eye, principally for its research design. In Citation Stickiness, Computer-Assisted Legal Research, and the Universe of Thinkable Thoughts, Aaron Kirschenfeld (UNC) and Alexa Chew (UNC), set out to explore how citation “stickiness” has survived the emergence of a “digital era” in legal research.

What the paper means by “stickiness” is that, in theory, the percentage of sticky citations   in   courts’   opinions   would   be   something   approaching 100%. That is, the  parties  would  discuss  the  relevant authorities in their briefs, and the court would rely  on  the  same  authorities  in  its  opinions.

Setting aside from the paper’s substantive contribution on “stickiness,” for me the paper’s main analytic move is to assess “stickiness” across dynamic legal research platforms over time. While the data are limited to a small handful of cases (N=325) from a single Circuit (CA4) and a handful of years (1957, 1987, and 2017), I anticipate that other research will employ a similar move to gain additional insights on how the emergence of the “digital era” informs law-related output. The abstract follows.

“Legal information has been available in widespread digital format for more than forty years. In that time, law librarians have wondered whether this digital switch has changed how law students and lawyers conduct research and, if so, what those changes are. Does legal research differ when conducted in print sources rather than computerized sources? What influence did the systems of organizing law in the print era have on the digital systems that followed?

While we cannot put these questions to rest, we hope to shed some light on those difference by studying the work of lawyers and courts from the print era, the transition-to-digital era, and the digital era. Using a metric called “citation stickiness,” we studied how often parties to an appeal and the judges hearing that appeal agreed on the cases relevant to resolve the issues on appeal. This, we hoped, would also show whether there were perceptible differences in coherence and stability of the legal information landscape

Citation stickiness works like this: a citation is “sticky” if it appears in at least one party’s brief and then again in the court opinion. In an initial study of 325 federal court cases from 2017, 49% of the 7,552 cases that were cited in the courts’ opinions had been cited by at least one party in a brief.

This study considers cases from the United States Court of Appeals for the Fourth Circuit decided in 1957, 1987, and 2017. After examining the citations in the briefs and opinions in our sample of cases, we compare our findings from the pre-digital era and from the digital era. What we learned surprised us.”

Adjusting for "Ambiguous" Zeros in an Ordered Logit Model

In the ELS silo, what to do with--and/or how to best interpret--"zeros" frequently persist as common challenges. In the world of standard count data, possible approaches include Zero-inflated Poisson Regression and Zero-inflated Negative Binomial Regression. Ordered logit analyses, however, require something slightly different.

To illustrate, a standard example is social science research (and explicated in this brief yet helpful video) includes, e.g., a four-siloed response to the question "How many cigarettes did you smoke each day, on average (0, 1-7, 8-12, >12)?" The possible ambiguity of the "0" response is that it may include responses from people who smoked less than one cigarette per day, on average, as well as those do not smoke cigarettes at all. And if the research question is designed to get a smokers' daily volume, the "0" response from non-smoker respondents is qualitatively different than a "0" response from smokers.

To address potential distortions to the "0" response introduced by non-smokers in this hypothetical, Stata's ziologit command fits a model for ordinal outcomes with an expected "excess" of "0" responses.

An Updated Database of State Tort Law Reforms (7th)

Ronen Avraham (Tel Aviv Univ./Texas) recently alerted me to an update of one of the more helpful resources around for those who study torts and tort reform. Specifically, the Database of State Tort Law Reforms (7.0), seeks to create "one 'canonized' dataset will increase our understanding of tort reform’s impacts on our lives." A fuller (though excerpted) description follows.

"This This manuscript of the DSTLR (7th) updates the DSTLR (6th) and contains the most detailed, complete and comprehensive legal dataset of the most prevalent tort reforms in the United States between 1980 and 2020.... The dataset records state laws in all fifty states and the District of Columbia over the last several decades. For each reform we record the effective date, a short description of the reform, whether or not the jury is allowed to know about the reform, whether the reform was upheld or struck down by the states’ courts, as well as whether it was amended by the state legislator. "

Pretrial Detention and Release Experiments

As I have an admitted soft-spot for (even quasi-) randomized control trials as a research design, a recent paper by Javier Osorio (Arizona-Govt. & Pub Pol’y) et al., Reducing Pretrial Detention: A Randomized Intervention with Public Defenders in El Salvador, caught my eye. In it, the authors exploit a "randomized intervention in El Salvador designed to improve public defenders’ capacities to secure pretrial release for defendants.”

To note that pretrial detention is a problem is to note merely the obvious. And in countries facing acute security challenges, such as El Salvador, there exists enormous pressure on criminal justice authorities to limit pretrial release. The authors contend that the overreliance on pretrial detention is driven in part by lack of information about defendants’ flight risks and the prospect of their committing further crimes while awaiting trial. And this information deficit is “the consequence of deficiencies in the public defender system, upon which economically disadvantaged citizens rely to ensure legal representation.”

To assess their hypothesis, the authors exploit an unusual (by U.S. standards, anyway) experiment involving a randomly assigned treatment to public defenders in four municipalities in El Salvador. “The treatment included specialized legal training, an improved interview protocol, and enhanced communication channels with the defendants, their relatives, and other institutions. These components sought to improve public defenders’ ability to gather evidence that would support pretrial release.”

The authors found that the treatment “increased public defenders’ requests for pretrial release by nearly 10%. The intervention also helped public defenders to secure pretrial release of their clients by 4.4%, although these results are weaker and sensitive to model specification.” The paper's abstract follows.

"Approximately one third of the global prison population is in pretrial detention, waiting for trial. Overreliance on pretrial detention exposes defendants to harsh conditions, exacerbates jail overcrowding, increases recidivism, and favors criminal governance. What policies can resource strapped countries implement to effectively address excessive pretrial detention? Based on a theoretical model focused on institutional-level efforts, we evaluate an experimental intervention implemented in El Salvador intended to increase pretrial release requests and reduce pretrial detention. The intervention randomly assigned public defenders to receive specialized legal training, an improved interview protocol, material resources, and increased communication channels. We find that this inexpensive, scalable program increased pretrial release requests from public defenders by nearly 10% (0.228 standard deviations) and increased the success in securing pretrial release by 4.4% (0.114 standard deviations). Heterogeneous treatment effect analyses suggest that the program increased strategic litigation among the most experienced public defenders and has distinct effects for those accused of minor and severe crimes. We find no evidence that the mechanism explaining our results involves changes in public defenders’ attitudes or perceptions about their work environments. Criminal justice programs focusing on pretrial detention may help reduce prison overcrowding in high crime countries."