Empirical Legal Studies

15th Annual Conference on Empirical Legal Studies (CELS)

Given the many disruptions incident to the COVID-19 pandemic, I am nothing short of delighted to announce that the Fifteenth Annual Conference on Empirical Legal Studies (CELS), presented by the Society for Empirical Legal Studies (SELS) and hosted by the University of Toronto Faculty of Law (Canada), is now scheduled for Friday, March 18, and Saturday, March 19, 2022.

CELS brings together scholars from an array of fields, including law, economics, political science, psychology, statistics, sociology, history, and policy analysis. Papers are selected through a peer-review process and each is assigned an individual discussant. CELS invites empirical papers that span all areas of empirical legal studies. Authors are encouraged to submit works-in-progress; however, submissions should be completed paper drafts that include main results. Submitted papers must also be unpublished (and are expected to be unpublished at the time of the conference). The paper submission portal will open August 1, 2021. The deadline for submissions is October 17, 2021.

Of course, while the conference organizers plan on (and sincerely hope for) an in-person event, given the ongoing uncertainties participants are urged to delay purchasing any non-refundable airline tickets at this moment. Additional conference information is found here.

A Descriptive Glance at Police Clearance Rates

While I have an admitted affinity for asking simple questions, sometimes it's the "simple" question that can help point the way toward more sophisticated empirical work. A recent paper by Andrew Leipold (Ill.), The Puzzle of Clearance Rates, and What They Can Tell Us About Crime, Police Reform, and Criminal Justice, makes my point.

The paper begins by framing its core research question: "Recent incidents of police violence have led to widespread reform efforts, from modest proposals to change police practices to dramatic attempts to slash funding or abolish the police entirely. But largely ignored in the debate is a simple question – how well is law enforcement currently performing its core functions? In particular, how good are the police at finding the perpetrator, arresting that person, and gathering enough evidence to start the matter through the criminal justice system?"

Exploring police clearance rate data engages the question. And what Leipold finds is that "Clearance rates for violent and property crimes have been both quite low and amazingly steady for the last 40 years. These figures are counterintuitive, because during that same period, crime first rose and then decreased dramatically; law enforcement personnel numbers increased, and then flattened; and the legal enforcement landscape appears to have tilted in the direction of the police and prosecution. Each of these changes should have significantly affected the clearance rates, but even collectively, they did not."

The presentation of basic descriptive trend data amply supports Leipold's point. And what he argues is that "[w]hile there is no single explanation, the diversion of resources from index crimes and a change in focus toward crime prevention seem to have the most explanatory power." While how to interpret--or explain--the counterintuitive trends, of course, remains ripe for future work, part of the power of initial descriptive work is that it helps bring into sharper relief avenues for deeper and more methodologically sophisticated future research.

The Empirics of Rule 23(f)

While civil procedure scholars have spilled a lot of ink on class actions, until quite recently little of it is informed by data. To help remedy this, in a recent paper, Class Certification in the U.S. Courts of Appeals: A Longitudinal Study, Stephen Burbank (Penn) and Sean Farhang (Berkeley) exploit a class certification decision data set that includes USCA precedential panel decisions (1966 through 2017) and nonprecedential panel decisions (2002 through 2017). The data permit the authors to test various "conventional wisdoms," including those related to class certification appeal outcomes and Rule 23(f)'s potential influence. Among their findings includes, e.g., that, "contrary to conventional expectations, in the period since Wal-Mart and Comcast, plaintiffs have been winning certification appeals more frequently than they were formerly, and Rule 23(f) contributed to this recent success." More specifically, the paper notes "significant variation over time in appeal outcomes under Rule 23(f), with defendants far more successful than plaintiffs prior to Wal-Mart and Comcast, and relative parity after." An excerpted abstract follows.

“There is a vast literature on the modern class action, but little of it is informed by systematic empirical data. Mindful both that there have been few Supreme Court class certification decisions and that they may not provide an accurate picture of class action jurisprudence, let alone class action activity, over time, we created a comprehensive data set of class certification decisions in the United States Courts of Appeals consisting of all precedential panel decisions addressing whether a class should be certified from 1966 through 2017, and of nonprecedential panel decisions from 2002 through 2017.

…

We find significant variation over time in appeal outcomes under Rule 23(f), with defendants far more successful than plaintiffs prior to Wal-Mart and Comcast, and relative parity after. This variation suggests the hazards of generalizing about operation of that rule from experience in any particular period. Our models also show that, for reasons about which we can only speculate, interlocutory appeals since around 2000 have elicited more ideological voting behavior by judges, leading to greater polarization.”

Count data, Logging Zeros, and Poisson

A persistent issue confounding many empirical legal studies using count data involves the "zeros problem," especially when the count data require a log transformation. Contests on how to best deal with this problem persist. Popular approaches include standard OLS regression models that add a "plus one" to each count data so to not lose all cases with a zero count in the log transformation and Poisson regressions models.

While empirical assessments of these two approaches are relatively rare, a recent paper compares these two approaches drawing on replication analyses of five recent corporate patenting papers. In Count Data in Finance, Jonathan Cohn (Texas--Business) et al., find that one OLS regression approach (LOG1PLUS) "using log-transformed outcomes can produce biased and incorrectly signed estimates of economic relationships and provide guidance for future research in finance-related applications involving zero-bounded count data." By comparison, while the Poisson model estimates may "lose efficiency if the model’s conditional mean-variance equality restriction is not satisfied in the data, they remain unbiased and consistent as long as the standard conditional mean independence assumption holds." The paper's abstract follows.

"This paper examines the use of count data-based outcome variables such as corporate patents in empirical corporate finance research. We demonstrate that the common practice of regressing the log of one plus the count on covariates (“LOG1PLUS” regression) produces biased and inconsistent estimates of objects of interest and lacks meaningful interpretation. Poisson regressions have simple interpretations and produce unbiased and consistent estimates under standard exogeneity assumptions, though they lose efficiency if the count data is overdispersed. Replicating several recent papers on corporate patenting, we find that LOG1PLUS and Poisson regressions frequently produce meaningfully different estimates and that bias in LOG1PLUS regressions is likely large."

Life Sentences "With" or "Without"

While significant public and scholarly attention rightly dwells on capital punishment, comparatively under-explored is a steady surge of life sentences without the possibility of parole ("LWOP"). While the Furman Court's "death is different" doctrine retains obvious salience and moral force, it should not obscure LWOPs from much-needed empirical inquiry.

To this end, in a recent paper, Life 'With' or 'Without'?: An Empirical Study of Homicide Sentencing, forthcoming in JELS, Michael O'Hear (Marquette) and Darren Wheelock (Marquette--Dept. Social and Cultural Sciences) do just that. While the paper's data set is limited to Wisconsin homicides, the authors do a deep data dive into 450 individuals convicted of first-degree intentional homicide between the start of 2001 and about mid-2018. The core findings are mixed. On the one hand, "intentional-homicide defendants who have more serious criminal histories are more likely to receive LWOP." On the other hand, "sentencing outcomes are influenced by the identity of the judge and prosecutor." The paper's abstract follows.

"The number of Americans serving sentences of life without the possibility of parole (“LWOP”) has grown rapidly over the past generation and now exceeds 50,000. Yet, little empirical research has been conducted on the determinants of LWOP sentences. The dearth of research on LWOP sentencing stands in sharp contrast to the many dozens of studies that have been conducted on the determinants of death sentences—studies that have consistently found that race, gender, and other questionable factors may influence sentencing outcomes. The present study is the first to employ a similar methodology to identify both case- and county-level variables that are correlated with the imposition of discretionary LWOP sentences.

More specifically, we have assessed the relationship between fifty different variables and LWOP decisions in 450 homicide cases in Wisconsin between 2001 and 2018. In our final model, we find seven variables that are correlated with sentencing outcomes. Of particular note, we find that judge and prosecutor personal characteristics are statistically significant correlates of LWOP decisions. We also find a significantly greater likelihood that LWOP sentences will be imposed in counties that are more Republican. We conclude with a proposal for a new LWOP sentencing process that may help to ensure that this very severe sentence is reserved for the most serious crimes committed by the most dangerous defendants."

"Results-to-Table" in Stata

While Stata--to my mind, anyway--still has not "fixed" the "unnecessary" degree of complexity in its graphics capabilities, it's nice to see it attend to the critical and often-challenging "results-to-table" migration task in v.17. While some may simply "hand-transcribe" results from the computer screen onto a table created for or embedded into Word documents, the more adventurous likely play with various "outreg" and related commands (e.g., "outreg2"). Stata has recently turned its attention to an alternative (or complementary) approach with updated commands that facilitate users "customizing" table presentations derived from various results. Unlike most efforts thus far, coding for this approach is "native" to Stata. Again, while I'm not entirely sure whether Stata has, once again, made the process unduly complicated, for anyone interested an introductory (and explanatory) video is here.

"Judging" Autonomous Vehicle Liability

Conventional wisdom implies that, on balance, judges, as a group and in their professional capacity, tend to disfavor new technologies. This judicial lurch towards disfavoring can present important policy implications as, especially in common law systems, judges can play a critical role in the regulation of new technologies. One example that will likely play out soon in real-time involves autonomous cars.

Dominating the experimental judicial decisionmmkaing literature are studies that rely on "mock" judges deciding "experimental" case hypotheticals. All too often, however, these "mock" judges involve either undergraduate students or a non-random collection of MTurk respondents. By contrast, findings from a recent paper, Judging Autonomous Vehicles, by Jeff Rachlinski (Cornell) and Judge Andrew Wistrich (Ca Central Dist. Ct), both valued Cornell colleagues, reflect an important advancement insofar as they derive from actual, real-live sitting judges (N=933).

In their studies, the authors presented hypothetical cases in which judges had to assess fault or determine damages for either an accident caused by an autonomous vehicle or by a human driver. What the paper finds is that "judges will treat automated vehicles with skepticism. In the related experiments we conducted, judges treated an autonomous vehicle that had been involved in a car accident more harshly than a human-driven vehicle involved in an essentially identical accident.” As these results comport with others prior research focusing on other technologies, larger policy implications loom. As the authors note, "at least when an innovation is artificial, non-human, and a dramatic departure from existing practices, judges act as a brake, delaying and hindering its adoption." The paper's abstract follows.

“The introduction of any new technology challenges judges to determine how it into existing liability schemes. If judges choose poorly, they can unleash novel injuries on society without redress or stifle progress by overburdening a technological breakthrough. The emergence of self-driving, or autonomous, vehicles will present an enormous challenge of this sort to judges, as this technology will alter the foundation of the largest source of civil liability in the United States. Although regulatory agencies will determine when and how autonomous cars may be placed into service, judges will likely play a central role in defining the standards for liability for them. How will judges treat this new technology? People commonly exhibit biases against innovations such as a naturalness bias, in which people disfavor injuries arising from artificial sources. In this paper we present data from 933 trial judges showing that judges exhibit bias against selfdriving vehicles. They both assigned more liability to a self-driving vehicle than they would to a human-driven vehicle and treated injuries caused by a self-driving vehicle as more serious than injuries caused by a human-driven vehicle.”

Diff-in-Diff

Owing to the steadily increasing popularity of Diff-in-Diff models across an array of fields, including law, Stata's newest version (v.17) builds out its Diff-in-Diff modeling capacities (a brief summary tutorial video is here). While Stata's brief video provides a quick introduction, Brady Neal's more detailed Diff-in-Diff modeling video (here) might assist those seeking more comprehensive information. Obviously, Diff-in-Diff models are not without important limitations, including the all-important parallel trends assumption. That such modeling limitations exist, however, do not distinguish Diff-in-Diff specifications from other competing empirical strategies.

Misdemeanor (Non) Prosecution

Hidden in plain sight is a significant amount of criminal misdemeanor activity in state courts. Indeed, "every year approximately 13 million Americans are charged with misdemeanor offenses, and misdemeanor cases make up over 80 percent of the cases processed by the U.S. criminal justice system." Scholars and policymakers increasingly raise concerns with the sheer volume of prosecutions for low-level offenses. As well, "some district attorneys across the country have begun to implement alternatives to misdemeanor prosecution, particularly for nonviolent defendants."

As scholarly attention continues to focus on criminal misdemeanors (see, e.g., here), it is inevitable that larger data sets and more sophisticated analyses emerge. One such example includes a recent paper, Misdemeanor Prosecution. In it, Agan (Rutgers--econ) et al., assess a practice in the Suffolk County (MA) District Attorney's Office that imposes upon prosecutors a "presumption of nonprosecution" for "nonviolent misdemeanor offenses." To do so, the paper exploits "new data [N=67,553] on the prosecution of nonviolent misdemeanor criminal complaints from the Suffolk County District Attorney’s Office (SCDAO) in Massachusetts between 2004 and 2018 to estimate the impact of nonprosecution of nonviolent misdemeanors on future criminal justice system contact." (emphasis added)

Despite some missing ADA-level data concerns, one main result is that: "increasing nonprosecution reduced the likelihood of subsequent criminal complaints within a one-year post-arraignment window. In addition, there does not appear to have been an increase in reported crime due to the policy change." The abstract follows.

"Communities across the United States are reconsidering the public safety benefits of prosecuting nonviolent misdemeanor offenses. So far there has been little empirical evidence to inform policy in this area. In this paper we report the first estimates of the causal effects of misdemeanor prosecution on defendants' subsequent criminal justice involvement. We leverage the as-if random assignment of nonviolent misdemeanor cases to Assistant District Attorneys (ADAs) who decide whether a case should move forward with prosecution in the Suffolk County District Attorney's Office in Massachusetts. These ADAs vary in the average leniency of their prosecution decisions. We find that, for the marginal defendant, nonprosecution of a nonviolent misdemeanor offense leads to large reductions in the likelihood of a new criminal complaint over the next two years. These local average treatment effects are largest for first-time defendants, suggesting that averting initial entry into the criminal justice system has the greatest benefits. We also present evidence that a recent policy change in Suffolk County imposing a presumption of nonprosecution for a set of nonviolent misdemeanor offenses had similar beneficial effects: the likelihood of future criminal justice involvement fell, with no apparent increase in local crime rates."

More Stata Plotting Options

The newly-released subsetplot command produces an array of scatter or other twoway plots for sub-groups of data. As Nick Cox observes (here): "We all know that if you want to compare relationships graphically between groups of observations, we can superimpose different groups in a single plot, or juxtapose different groups in several plots. This [the subsetplot command] is a hybrid approach combining elements of those two strategies."

To illustrate, consider the following plot of auto weight by MPG:The subsetplot command plots these same data across a selected sub-grouping's outcomes (below, by auto repair record groupings (1-5)):For those interested, to download the [Stata] command type:

ssc install subsetplot

Data on Insider "Giving"

When corporate insiders learn about material, non-public bad news at their company that devalues stock prices, they can avoid losses by selling their stock before the public finds out. This form of “insider trading,” is, of course, illegal and risky. But selling stock is not the only way for corporate insiders to cash out. One alternative strategy, which is almost as financially effective and benefits from comparatively less onerous reporting requirements and legal restrictions, involves donating the stock to a charity and taking a tax deduction at the stock's inflated price. While a financial "second-best" strategy, "insider giving" nonetheless endures as a plausibly potent substitute for insider trading.

In Insider Giving, Sureyya B. Avci (Sabanci Univ.--Business) et al., submit their "insider giving" hypothesis to data. As the paper describes, the dataset includes "all reported gifts of common stock and contains over 9,000 observations between 1986 and 2020. The total volume of gifts contained in our dataset is approximately 2.1 billion shares, with a dollar value of approximately $50 billion." And what they find is that "large shareholders’ gifts are suspiciously well timed. Stock prices rise abnormally about 6% during the one-year period before the gift date and they fall abnormally by about 4% during the one year after the gift date, meaning that large shareholders tend to find the perfect day on which to give."

Rather than ascribe these finding to insider-donors' incredible good luck, the paper instead develops two possible alternative explanations: "information leakage" and "backdating." Specifically, "executives seem to provide large shareholders with material non-public information, who then use it to time gifts. We also find a second explanation to be supported, though its magnitude is smaller – backdating. The telltale sign of backdating is that the givers’ extraordinary luck tends to grow alongside the delay they take in reporting the gift. A donor who waits a few weeks to report a gift can cherry pick the very best date to retroactively claim their gift was consummated. That is precisely what we find." An excerpted abstract follows.

"Corporate insiders can avoid losses if they dispose of their stock while in possession of material, non-public information. One means of disposal, selling the stock, is illegal and subject to prompt mandatory reporting. A second strategy is almost as effective and it faces lax reporting requirements and legal restrictions. That second method is to donate the stock to a charity and take a charitable tax deduction at the inflated stock price. “Insider giving” is a potent substitute for insider trading. We show that insider giving is far more widespread than previously believed. In particular, we show that it is not limited to officers and directors. Large investors appear to regularly receive material non-public information and use it to avoid losses. Using a vast dataset of essentially all transactions in public company stock since 1986, we find consistent and economically significant evidence that these shareholders’ impeccable timing likely reflects information leakage. We also document substantial evidence of backdating – investors falsifying the date of their gift to capture a larger tax break….”

6th Annual TAU Workshop for Junior Scholars--Call For Papers (and Optimism)

With the shared hope that some semblance of "healthy normalcy" will return to all sometime soon, I pass along the following Call For Papers.

The Tel Aviv University Buchmann Faculty of Law is pleased to invite submissions to its sixth junior legal scholar workshop. The workshop provides junior scholars with the opportunity to present and discuss their work, receive meaningful feedback from faculty members and peers, and actively participate in an international community of junior legal scholars.

"The upcoming 2021 TAU workshop for junior scholars in law, which will be held in Tel-Aviv University Law School on November 14-16, 2021, will be dedicated to 'Legal Change in Revolutionary Times.' We are seeking cutting-edge works in progress, in varied legal fields and written from a variety of methodological and theoretical perspectives, that investigate the role of law in revolutionary times, and we welcome works that broadly relate to the core themes of the workshop."

Conference organizers expressly welcomes proposals from: "doctoral candidates, and recent graduates of doctoral programs submissions. Participants from the global South and under-represented communities are particularly encouraged to apply." Paper proposals are due to conference organizers by June 1, 2021. More conference specific information can be found (here); those wishing to contact conference organizers directly can send an email to: TAU.junior.scholars@gmail.com

Finally, conference organizers have adopted an "optimistic" assumption and, at present, anticipate a traditional, "in-person" conference. They also note, however, that if "Covid-19 is till ruling our lives by November 2021," the conference workshop format will transform into a "hybrid-virtual."

What Would Happen if Plea Bargaining Was Banned?

For an array of reasons, plea bargaining endures as a concern for many criminal law scholars. Proponents of plea bargaining "point to its efficiency−since trials are costly if both sides form reasonably similar beliefs about the outcome to arise at trial they can just agree to that expected sanction beforehand and save the costs." Opponents, by contrast, fear that excessive plea bargaining rates--combined with various structural differences that distinguish many defendants (especially indigent ones) and prosecutors' offices--may help coerce some factually innocent defendants into pleading guilty rather than defending their innocence at trail.

Amid growing calls to "regulate the plea bargaining process in an attempt to reduce its prevalence," one state--Alaska--acted. Specifically, Alaska Attorney General Michael Geraghty announced a new policy limiting plea bargaining in July 2013. The limitation prohibits prosecutors from "negotiating over the length of the jail/prison sentence for those accused of committing violent crimes."

In a recent paper, Alaska's Plea Bargaining Ban, Bryan McCannon (Univ. West Vir.--Economics) sets out to assess the efficacy of Alaska's experiment. The paper exploits a data set that includes annual (2007-2019) court-year data on how all of Alaska's Superior Courts disposed of their criminal and civil dockets (N=874). That Alaska's plea bargaining ban can affect a variety of margins raises important identification concerns. One thing the ban cannot affect, however, are pretrial plea negotiations in civil cases. Thus, the paper compares changes in the civil and criminal contexts.

The primary variable of interest is the rate at which cases settled pretrial and the author estimates a diff-in-diff specification. The critical take-away from the findings is that Alaska's plea bargaining ban for defendants charged with violent criminal crimes "does not have a meaningful effect" (emphasis added). One explanation for the absence of any detected affect involves the (often-underappreciated) importance of prosecutors' charge-bargaining policies and practices (which fall outside of the Alaska's ban on negotiating jail/prison sentences).

To be sure, important study limitations exist. Alaska may be an anomalous state. As well, as the study draws from an aggregated panel data set it cannot account for various case-specific factors. Finally, aggregating the case outcome data to the court level invites ignoring various heterogeneous treatment effects. Research design limitations aside, however, while this study will certainly not be the final word on this topic it is a helpful first step. The paper's abstract follows.

“Plea bargaining dominates the U.S. criminal justice system and has garnered calls to reduce its prevalence. In 2013 Alaska’s Attorney General acted banning a practice known as sentencing bargaining where the prosecutor negotiates with the defense over the length of the incarceration. I provide the first causal identification of this policy’s impact on the plea bargaining rate. I show that the policy, by leaving open the charging discretion (in effect allowing charge bargaining), was ineffective at changing plea bargaining’s prevalence across the state. Policymakers should look elsewhere for tools if they want to mitigate its rate.”

"Count" Variables & Dispersion

In many ELS contexts, count data are noted over-dispersion. In many legal settings this issue presents in the form of a "zeros-problem." When such a "skewed, count" dependent variable arises, various approaches include: negative binomial, Poisson, and zero-inflated regression estimations. A recent discussion of this general topic on the StataList (here) includes a robust back-and-forth about the relative strengths and weaknesses of various approaches. The claim that Poisson regression is not inappropriate when there is overdispersion in the dependent variable is forcefully defended. (For an earlier discussion of this general topic click here.)

A New and Interesting Take on Measuring Case Complexity

Most judicial decisionmaking papers can be criticized for inadequately controlling for variation in "case complexity." Lingering disputes over what "complexity" even means in terms of litigated cases only increase the degree of difficulty in constructing such a control. And even if a consensus on what "case complexity" means existed, constructing a reliable, consistent, and stable measure introduces additional challenges.

Difficulties, conceptual and mechanical, aside, a recent paper, Measuring Supreme Court Case Complexity, makes a helpful contribution. In it, Greg Goelzhauser (Utah State--Poli Sci) et al., both critique an array of earlier efforts and advance their own Bayesian measurement model to construct a latent (Supreme Court) case complexity measure that exploits issue and provision counts from the cases' merits briefs. By measuring case complexity from pre-decision data (merits briefs) the paper argues that its ex poste approach "diminishes endogeneity and post-treatment bias concerns when studying merits stage outcomes." To develop its measure, the authors draw from issue and provision (hand-coded) counts extracted from merits briefs filed in the 1954 through 2017 Court Terms, incident to Supreme Court Rules 24(1)(a) and 24(1)(f).

While perhaps limited to Supreme Court cases, new approaches to a difficult challenge can make important methodological contributions. The paper's abstract follows.

"Case complexity is central to the study of judicial politics. The dominant measures of Supreme Court case complexity use information on legal issues and provisions observed post-decision. As a result, scholars using these measures to study merits stage outcomes such as bargaining, voting, separate opinion production, and opinion content introduce post-treatment bias and exacerbate endogeneity concerns. Furthermore, existing issue measures are not valid proxies for complexity. Leveraging information on issues and provisions extracted from merits briefs, we develop a new latent measure of Supreme Court case complexity. This measure maps with the prevailing understanding of the underlying concept while mitigating inferential threats that hamper empirical evaluations. Our brief-based measurement strategy is generalizable to other contexts where it is important to generate exogenous and pre-treatment indicators for use in explaining merits decisions."