Empirical Legal Studies

The Federal Circuit Dataset project (v.3)

While Jason Rantanen's (Iowa) third update to The Federal Circuit Dataset Project will certainly interest empirical IP scholars, the Project's broader implications speak to empirical law scholars in all fields.

The third edition advances the existing Compendium of Federal Circuit Decisions in at least two main ways. First, the dataset now includes document numbers for all archived documents. Second, the dataset now facilitates increased understanding about what happens to appeals, including information about appeal pendency and origin.

Setting aside the dataset's utility to empirical IP scholars, the Project itself reflects a growing commitment to the hard and often under-rewarded work of dataset creation as well as an impulse to make such data (and related codebooks, etc.) publicly-available. Such impulses need to be rewarded, promoted, and nurtured. The paper's abstract follows.

"This paper introduces the third generation of the Federal Circuit Dataset Project. The third generation builds on the existing Compendium of Federal Circuit Decisions with the following enhancements:

• The addition of a dataset of all dockets for the Federal Circuit since 2000. These dockets are linked to the documents in the Compendium through the Appeal_Dockets field in both datasets.
• New fields containing additional information about Federal Circuit decisions.
• More complete data in existing fields that require manual coding.
• Parallel archiving of the document and docket datasets on the Dataverse.

In addition to describing the third generation of the Federal Circuit Dataset Project, this paper also provides updated descriptive statistics about Federal Circuit decisions and new statistics on appeal dockets."

Technology and Legal Citation "Stickiness"

An interesting citation paper caught my eye, principally for its research design. In Citation Stickiness, Computer-Assisted Legal Research, and the Universe of Thinkable Thoughts, Aaron Kirschenfeld (UNC) and Alexa Chew (UNC), set out to explore how citation “stickiness” has survived the emergence of a “digital era” in legal research.

What the paper means by “stickiness” is that, in theory, the percentage of sticky citations   in   courts’   opinions   would   be   something   approaching 100%. That is, the  parties  would  discuss  the  relevant authorities in their briefs, and the court would rely  on  the  same  authorities  in  its  opinions.

Setting aside from the paper’s substantive contribution on “stickiness,” for me the paper’s main analytic move is to assess “stickiness” across dynamic legal research platforms over time. While the data are limited to a small handful of cases (N=325) from a single Circuit (CA4) and a handful of years (1957, 1987, and 2017), I anticipate that other research will employ a similar move to gain additional insights on how the emergence of the “digital era” informs law-related output. The abstract follows.

“Legal information has been available in widespread digital format for more than forty years. In that time, law librarians have wondered whether this digital switch has changed how law students and lawyers conduct research and, if so, what those changes are. Does legal research differ when conducted in print sources rather than computerized sources? What influence did the systems of organizing law in the print era have on the digital systems that followed?

While we cannot put these questions to rest, we hope to shed some light on those difference by studying the work of lawyers and courts from the print era, the transition-to-digital era, and the digital era. Using a metric called “citation stickiness,” we studied how often parties to an appeal and the judges hearing that appeal agreed on the cases relevant to resolve the issues on appeal. This, we hoped, would also show whether there were perceptible differences in coherence and stability of the legal information landscape

Citation stickiness works like this: a citation is “sticky” if it appears in at least one party’s brief and then again in the court opinion. In an initial study of 325 federal court cases from 2017, 49% of the 7,552 cases that were cited in the courts’ opinions had been cited by at least one party in a brief.

This study considers cases from the United States Court of Appeals for the Fourth Circuit decided in 1957, 1987, and 2017. After examining the citations in the briefs and opinions in our sample of cases, we compare our findings from the pre-digital era and from the digital era. What we learned surprised us.”

Adjusting for "Ambiguous" Zeros in an Ordered Logit Model

In the ELS silo, what to do with--and/or how to best interpret--"zeros" frequently persist as common challenges. In the world of standard count data, possible approaches include Zero-inflated Poisson Regression and Zero-inflated Negative Binomial Regression. Ordered logit analyses, however, require something slightly different.

To illustrate, a standard example is social science research (and explicated in this brief yet helpful video) includes, e.g., a four-siloed response to the question "How many cigarettes did you smoke each day, on average (0, 1-7, 8-12, >12)?" The possible ambiguity of the "0" response is that it may include responses from people who smoked less than one cigarette per day, on average, as well as those do not smoke cigarettes at all. And if the research question is designed to get a smokers' daily volume, the "0" response from non-smoker respondents is qualitatively different than a "0" response from smokers.

To address potential distortions to the "0" response introduced by non-smokers in this hypothetical, Stata's ziologit command fits a model for ordinal outcomes with an expected "excess" of "0" responses.

An Updated Database of State Tort Law Reforms (7th)

Ronen Avraham (Tel Aviv Univ./Texas) recently alerted me to an update of one of the more helpful resources around for those who study torts and tort reform. Specifically, the Database of State Tort Law Reforms (7.0), seeks to create "one 'canonized' dataset will increase our understanding of tort reform’s impacts on our lives." A fuller (though excerpted) description follows.

"This This manuscript of the DSTLR (7th) updates the DSTLR (6th) and contains the most detailed, complete and comprehensive legal dataset of the most prevalent tort reforms in the United States between 1980 and 2020.... The dataset records state laws in all fifty states and the District of Columbia over the last several decades. For each reform we record the effective date, a short description of the reform, whether or not the jury is allowed to know about the reform, whether the reform was upheld or struck down by the states’ courts, as well as whether it was amended by the state legislator. "

Pretrial Detention and Release Experiments

As I have an admitted soft-spot for (even quasi-) randomized control trials as a research design, a recent paper by Javier Osorio (Arizona-Govt. & Pub Pol’y) et al., Reducing Pretrial Detention: A Randomized Intervention with Public Defenders in El Salvador, caught my eye. In it, the authors exploit a "randomized intervention in El Salvador designed to improve public defenders’ capacities to secure pretrial release for defendants.”

To note that pretrial detention is a problem is to note merely the obvious. And in countries facing acute security challenges, such as El Salvador, there exists enormous pressure on criminal justice authorities to limit pretrial release. The authors contend that the overreliance on pretrial detention is driven in part by lack of information about defendants’ flight risks and the prospect of their committing further crimes while awaiting trial. And this information deficit is “the consequence of deficiencies in the public defender system, upon which economically disadvantaged citizens rely to ensure legal representation.”

To assess their hypothesis, the authors exploit an unusual (by U.S. standards, anyway) experiment involving a randomly assigned treatment to public defenders in four municipalities in El Salvador. “The treatment included specialized legal training, an improved interview protocol, and enhanced communication channels with the defendants, their relatives, and other institutions. These components sought to improve public defenders’ ability to gather evidence that would support pretrial release.”

The authors found that the treatment “increased public defenders’ requests for pretrial release by nearly 10%. The intervention also helped public defenders to secure pretrial release of their clients by 4.4%, although these results are weaker and sensitive to model specification.” The paper's abstract follows.

"Approximately one third of the global prison population is in pretrial detention, waiting for trial. Overreliance on pretrial detention exposes defendants to harsh conditions, exacerbates jail overcrowding, increases recidivism, and favors criminal governance. What policies can resource strapped countries implement to effectively address excessive pretrial detention? Based on a theoretical model focused on institutional-level efforts, we evaluate an experimental intervention implemented in El Salvador intended to increase pretrial release requests and reduce pretrial detention. The intervention randomly assigned public defenders to receive specialized legal training, an improved interview protocol, material resources, and increased communication channels. We find that this inexpensive, scalable program increased pretrial release requests from public defenders by nearly 10% (0.228 standard deviations) and increased the success in securing pretrial release by 4.4% (0.114 standard deviations). Heterogeneous treatment effect analyses suggest that the program increased strategic litigation among the most experienced public defenders and has distinct effects for those accused of minor and severe crimes. We find no evidence that the mechanism explaining our results involves changes in public defenders’ attitudes or perceptions about their work environments. Criminal justice programs focusing on pretrial detention may help reduce prison overcrowding in high crime countries."

Visual Displays of Uncertainty

A recent post on Gelman's Stats blog (by Jessica Hullman) nicely merges a few critical threads. One involves Tufte's classic push to display data in creative and informative ways (see, e.g., Tufte's classic). Another relates to a detectable nudge towards emphasizing results' "uncertainty" rather than a "false" sense of "precision" through, say, point estimates. While Jessica's discussion dwells on election forecast uncertainty--and interesting ways to visually convey it--the basic point includes obvious spill-over implications. As well, even if one is persuaded that visually conveying uncertainty is a worthwhile objective, the question about whether withholding point estimates a good idea is not easily answered.

Ways to Approach (and "Interpret") Null Results

"Null" results are a species of output that remains comparatively under-explored. Or, more accurately, how much analytic weight null results can plausibly bear (and/or how to "interpret" null findings) quickly invite complex questions that frequently derail into other, related lines of inquiry (the very concept of "yes/no" statistical "significance", power testing, etc.) As is often the case, at bottom some level of judgment and discernment is required. This general topic was the subject of a recent StataList discussion (here).

Bringing Data to Firm Successions

It was perhaps inevitable that the popular success of the HBO series, Succession, might, in turn, help stimulate scholarly attention to related associated questions. As it relates to critical questions germane to "family firms'" stock and accounting performances, the general thrust in the academic literature, while mixed, trends in a negative direction. In Do Family Successions (Really) Reduce Firm Performance? Evidence from Large US Firms, Murali Jagannathan (SUNY Binghamton-, management) et al., take a fresh look with the benefit of a large dataset that includes "48,613 firm-years in 3,745 unique firms. Within our dataset, there are 683 total founder or founder-family successions, including 124 instances where the founder was succeeded by a family member.”

Two research design challenges complicate such "family firm" studies. First, the decision to appoint a founding family member as CEO is not random. To address this selection challenge, the paper adopts "an instrumental variables approach using two instruments: whether someone with the same last name as the founder serves on the board within the three-year period preceding the succession, and whether or not the firm has a long history.” Second, systematic firm differences distinguish "founder, founder family, and non-family firms." To adjust, the authors adopt a propensity match strategy to compare "founder family firms with non-founder firms and examine the change in performance around founder successions relative to the matched firms.”

Notably, the paper finds "no significant decreases in either accounting performance or firm value, and instead find evidence that both are improved.” The authors lean on evidence that "family successions are associated with higher levels of employee trust" to help account for their findings. The paper's abstract follows.

"This paper examines whether founder family successions lower performance in large US firms. We find no significant decreases in either accounting performance or firm value, and instead find evidence that both are improved. Consistent with this, we present evidence that family successions are associated with higher levels of employee trust, and that employee’s human capital is not as easily transferable at family firms. Consistent with the literature, we find that the decision to appoint a founder family member as a successor CEO is not random, and occurs in low-growth, yet profitable firms. To mitigate these endogeneity problems, we first propensity match founder family firms with non-founder firms and examine the change in performance around founder successions relative to the matched firms. We also use two instruments of founder family successions in our analysis."

Potential Research Projects

While The De-licensing of Occupations in the United States: A Shifting Trend?, by Robert Thornton (Lehigh—Business) et al., is a brief and purely descriptive paper, it identifies a timely topic and, in so doing, also identifies possible future empirical projects.

While a traditional focus for economists, “occupational licensing has begun to attract a growing and sometimes vocal stream of critics.” The standard set of criticisms are relatively well-rehearsed. “It has been alleged that too many occupations now require a license to practice and that the requirements for attaining a license in terms of costs and length of training are often excessive. Complaints have also been made that higher prices for the services provided by licensed practitioners are too often the result, and that the benefits to consumers in terms of higher quality are either small or nonexistent. Finally, it has been charged that excessive licensing has resulted in adverse effects on employment opportunities, intergenerational mobility, and worker mobility, especially for those with lower levels of education.”

While the state legislatures have been slow to respond to such critiques, since 2011 state legislative responses have accelerated. “The result has been that 35 occupations have become delicensed in a total of seven states over the past five years.” While this paper only sets out to identify where de-licensing efforts have succeeded, the paper’s sourcing also identifies possible avenues of empirical research down the road assessing, e.g., the costs and benefits incident to occupational licensing requirements and delicensing.

Tobacco Products, Taxes, and Substitution Effects

Variation in e-cigarette taxation supplies potentially rich research opportunities for studies of the on-the-ground effects of tax policy. In this spirit, The Effects of E-Cigarette Taxes on E-Cigarette Prices and Tobacco Product Sales: Evidence from Retail Panel Data, by Chad Cotti (Univ. Wisc-Oshkosh--economics) et al., estimates the effects of "e-cigarette taxes enacted in eight states and two large counties on e-cigarette prices, e-cigarette sales, and sales of other tobacco products." (One critical data limitation is the study's reliance on e-cigarette data that excludes sales through specialty vape shops and online sales.) The intersection of two core findings help frame what the authors define as a "detrimental unintended consequence of e-cigarette taxes."

One key finding is that "e-cigarette taxes are over-shifted to consumer prices, which is most likely in a market with high concentration." A second key finding is that "e-cigarettes are elastic goods and their use substantially reduces cigarette sales." However, "a 1% increase in e-cigarette taxes increases cigarette sales by about 0.14%.” (emphasis added) What these two findings imply is that even well-intentioned efforts to dampen e-cigarette use (especially among teens and young adults) through tax policy may well induce for some a migration from e-cigarettes to even more harmful cigarettes.

Data Resources Courtesy of the APSA Law & Courts Section

Our good friends and political science colleagues over at the APSA's Law & Courts Section recently updated the Section's website (click here). What is especially helpful is a sub-page (here) that provides links to the leading court data sets (SCOTUS, CA, district courts, as well as state supreme courts) along with related data assets (e.g., Martin-Quinn Scores, Judicial Common Space Scores).

Matthew McCubbins (1956-2021)

It is with sadness that I note the passing of a friend and colleague Matthew McCubbins.

Bridge Ideal Points--2021 Update

Michael Bailey (Georgetown--Govt.) has updated the Bridge Ideal Points estimates (click here) to cover years 1950 through 2020. These Ideal point estimates include the Supreme Court, presidential, and congressional ideology estimates and are designed to facilitate inter-institutional and cross-temporal comparisons.

While the Bridge Ideal Points compete somewhat with the Martin-Quinn scores in the literature, the two sets of ideology estimates correlate reasonably highly with one another. Despite the overlap, these two ideology estimates differ in two important ways.First, "Bridge Ideal Points are designed to be intertemporally comparable. Martin and Quinn scores assume the ideological cutpoints of Supreme Court cases are drawn from a common distribution across years, which makes them less comparable across time when the case load changes." Second, while Martin and Quinn scores only cover the Supreme Court justices, Bridge Ideal Points for justices are comparable to Bridge Ideal Points for presidents and members of Congress as well.

Regardless of one's preference, efforts to estimate justice ideology remain a staple in the judicial decisionmaking literature.

Journal of the Patent and Trademark Office Society ("JPTOS")--Call For Submissions

The Journal of the Patent and Trademark Office Society ("JPTOS"), a leading intellectual property law journal for over one hundred years, has recently returned from a publishing hiatus due to the pandemic and is eager to accept new article submissions. Moreover, the Journal’s incoming EIC, Timothy Hsieh (Okla City), a long-time ELS Blog reader, expressly noted that the Journal publishes “a fair amount of articles involving empirical analysis.”

JPTOS seeks articles involving intellectual property (patents, trademarks, copyrights, trade secrets, or hybrids between all such forms) or law & technology (blockchain, cryptocurrencies, NFTs, artificial intelligence, 3D printing, social media, privacy/surveillance, biotechnology, pharmaceuticals, COVID vaccine patents or patent policy and topics) and all related fields. JPTOS is one of the oldest intellectual property journals in the United States and for over a century has been at the forefront of various intellectual property debates.

Submissions may be directly emailed to: editor@jptos.org (or to the Journal’s Scholastica location: https://jptos.scholasticahq.com/). Additional details regarding submission guidelines can be found at the Journal’s official website (http://jptos.org).

15th Annual Conference on Empirical Legal Studies (CELS)

Given the many disruptions incident to the COVID-19 pandemic, I am nothing short of delighted to announce that the Fifteenth Annual Conference on Empirical Legal Studies (CELS), presented by the Society for Empirical Legal Studies (SELS) and hosted by the University of Toronto Faculty of Law (Canada), is now scheduled for Friday, March 18, and Saturday, March 19, 2022.

CELS brings together scholars from an array of fields, including law, economics, political science, psychology, statistics, sociology, history, and policy analysis. Papers are selected through a peer-review process and each is assigned an individual discussant. CELS invites empirical papers that span all areas of empirical legal studies. Authors are encouraged to submit works-in-progress; however, submissions should be completed paper drafts that include main results. Submitted papers must also be unpublished (and are expected to be unpublished at the time of the conference). The paper submission portal will open August 1, 2021. The deadline for submissions is October 17, 2021.

Of course, while the conference organizers plan on (and sincerely hope for) an in-person event, given the ongoing uncertainties participants are urged to delay purchasing any non-refundable airline tickets at this moment. Additional conference information is found here.