Empirical Legal Studies

Experimental Data on Insurance Contracts & Consumer Comprehension

While insurance contracts are a distinct sub-species of general contracts and subject to some insurance doctrine-specific interpretative canons, contract law generally assumes that reasonable consumers reasonably understand the contracts that they sign. Despite the uncomfortable fact that too many (most?) consumers don't even bother reading standard form contract language before signing, whether those consumers who do read contracts before signing accurately understand them endures as a separate research question.

In a recent paper, Read But Not Understood? An Empirical Analysis of Consumer Comprehension in Homeowners Insurance, Daniel Schwarcz (Minn.) et al. report results from an experiment assessing whether "providing excerpts from the dominant standard form homeowners insurance policy improves consumer understanding of coverage." Their main finding, on balance, tends to upend a pivotal contract interpretation assumption: "providing [homeowners insurance] policy language only moderately improved consumer understanding in some scenarios, while affirmatively decreasing accuracy in others." As the authors note, their findings "carry significant legal and regulatory implications."

As the experimental results derive from surveys (N=2,440) and lever a large on-line survey platform (Dynata), important external validity questions linger. Methodological questions and limitations notwithstanding, the paper's core results succeed in drawing attention to core insurance contract assumptions. An excerpted abstract follows.

"Through a series of survey-based experiments, we compare consumers’ general beliefs about homeowners insurance with their beliefs when provided with key policy excerpts. Our main finding is that providing policy language only moderately improved consumer understanding in some scenarios, while affirmatively decreasing accuracy in others. Respondents often struggled with partial reading or misinterpretation of policy provisions, especially when broad coverage grants were later restricted by specific exclusions—a common structural feature of insurance policies."

Public Input Data on SEC Rulemaking

Incident to its rulemaking activities, the SEC, similar to other agencies, solicits public input on proposed rules and reduces to formal memoranda SEC meetings with industry advocates (and lobbyists) seeking to influence final rules. The emergence of large language models (LLMs) enables scholars to assess possible relations between both forms of public input (comments and memoranda) and final rules. And this is precisely what Adam Pritchard (Michigan) et al., set out to do in their recent paper, SEC Rulemaking: An Empirical Analysis of Comments and Memoranda.

The paper draws from 482 rules, 1018 releases, 84,828 comments, and 5,700 memoranda, which are then linked (through Westlaw and Lexis) to any subsequent trial and appellate court activity. These data were then analyzed for "sentiment" with leading LLMs, including OpenAI, specifically GPT-3.5. A focus on rulemaking activity, at least with respect to SEC's proposed rules, is warranted owing to, at the very least, a detectable trend towards increased public commentary over time as the figure (below) implies.

Among this paper's key findings include that while comment volume and sentiment systematically inform final SEC rules, unsurprisingly, this influence varies across different types of commentators. Similarly, direct advocacy engagement with SEC officials--indicated by SEC memoranda--"is an important marker of proposed rules’ adoption, and if adopted, judicial challenge and survival." An excerpted abstract follows.

"... We find that public inputs contain valuable information regarding the probability that a rule is adopted, challenged, and survives judicial scrutiny. Longer average comments and greater disagreement among commenters correlate with a lower likelihood of rule survival, whereas an increase in the number of suggestions correlates with a greater survival likelihood. Rules affecting issuers are less likely to be adopted and more likely to face a court challenge, while rules affecting broker-dealers are more likely to survive judicial scrutiny. As for specific issues, comments emphasizing the burden on competition correlate with a lower likelihood of rule survival, comments raising constitutional questions reduce both the likelihood of rule adoption and survival conditional on a judicial challenge. Comments by academics have no statistically significant effect at any stage of the process...."

Selection Effects & Prof. George Priest (1947-2024)

A prior post noting Prof. George Priest's passing predictably emphasizes his enduring influence on the topic of selection effects in litigation. Among the many papers scheduled for publication in 42:3 Yale J. on Reg. (2025) and presented at a symposium (or 'Festschrift') just months before Priest's recent passing includes 'The Selection of Disputes' at Forty, by Daniel Klerman (USC) & Yoon-Ho Alex Lee (Northwestern). The Klerman & Lee paper (abstract below) complements, contributes to, and reflects Priest's enormous scholarly legacy.

“Priest and Klein's ‘Selection of Disputes for Litigation’ has been one of the most influential law review articles of all time. This Article reviews its contribution to legal scholarship. Priest and Klein's central and enduring contribution is the recognition that some cases are more likely to settle than others. It follows that litigated disputes are not a random sample of all disputes. This basic insight is true under nearly all litigation models and is also confirmed by a large body of empirical evidence. Priest and Klein's article is also famous for its prediction that, under certain conditions, the plaintiff trial win rate will tend to approach fifty percent. That prediction, however, is not supported by most other litigation models and has received only modest support from the empirical literature. Our citation analysis also suggests that 'The Selection of Disputes for Litigation' is one of the rare articles whose importance was both recognized almost immediately and whose influence has continued to grow over several decades.”

Contingent Fees & Access to Justice: Data From NYC

While "contingent fees are certainly not without their problems," recent analyses of NYC data imply that they contribute to increased equity in terms of access to justice "between rich and poor and between whites and racial minorities."

It turns out that since 1957, New York courts require contingent fee lawyers to file a "closing statement" for all cases involving "personal injury, property damage, or wrongful death." While typically confidential, these closing statements also include settlement amounts.

In Contingent Fees and Access to Justice, the authors, Eric Helland (Claremont McKenna--econ) and Dan Klerman (USC), reports results from their analyses of closing data "from 2003–2013 from the First Appellate Division, which has supervisory authority over lawyers in Manhattan and the Bronx."

To be sure, even these rich data (especially the settlement data) are not without limitations. After all, the data only involve eleven years, two NYC boroughs, and certain tort claims. Any data limitations aside, key findings include that "the poor make claims and recover at higher rates than the rich, and African Americans and Hispanics make claims at higher rates than whites." The paper's abstract follows.

"Using a unique dataset from New York City, this article shows that contingent fees seem to have, at least partly, solved the access-to-justice problem in tort litigation. People in poorer zip codes, in fact, make legal claims at a higher rate than those in wealthier areas. This greater propensity to seek and achieve legal redress is partly explained by the fact that poor people are more likely to be injured. In addition, the disparity between rich and poor manifests itself solely in smaller claims. For larger claims, rich and poor make claims at roughly the same rate. African Americans and Hispanics also make claims at a higher rate than whites, and this difference also attenuates when one controls for accident rates or focuses on larger claims."

Anticipated and Unanticipated Costs and Benefits of Mandatory Insider Trading "Blackout" Requirements--Evidence From the EU

Many countries, including the U.S., impose closed periods (a.k.a. "blackouts") on corporate insider trading for a period of time preceding earnings announcements. Such regulations on insiders' trading activity are typically justified due to insiders' unique role and position in capital markets. Whether these regulations achieve their stated goals remains relatively understudied.

In July 2016, all E.U.-listed firms were required to impose a 30-day blackout period on corporate insider trading prior to interim or fiscal year-end earnings announcements. The variation across firms prior to the July 2016 rule created a quasi-natural experiment to assess the new (2016) rule's impact on "insider trading activity, profitability, and market consequences."

In The Impact of Mandatory Closed Periods on Corporate Insider Trading, Francois Brochet (BU—Accounting) et al. report results from a difference-in-differences strategy that compares 30 compare countries that adopted mandated closed periods after July 2016 (treatment group) with companies that already had such policies (control group) in place prior to 2016. Notably, what the authors report is that their results highlight that uniform closed periods "deter informed trading but may unintentionally harm market liquidity and information quality." The abstract follows.

"The Market Abuse Regulation (MAR), implemented in July 2016 across the EU, mandates uniform closed periods, prohibiting insider trading 30 days before earnings announcements. We find a significant reduction in insider trading during these periods in treated countries without prior mandates. Although some trades still occur, they do not appear to be driven by private information. However, treated insiders experience an increase in compensation. Furthermore, information asymmetry rises before earnings announcements, and treated firms experience a decline in institutional ownership. Overall, the study suggests that mandated closed periods do not lead to a wealth transfer from insiders to outsiders, nor do they enhance the information environment. These results underscore the limitations of regulatory one-size-fits-all measures in curbing corporate insider trading."

Empirical Evidence, Courts, and the 4th Amendment

Over time, and since Katz (1954), Justice Harlan's two-part approach has largely guided how courts construe privacy protections in the 4th Amendment context. As the second part of Harlan's approach involves what many consider to be amenable to empirical social science, a growing body of scholarship sets out to mark the boundaries of where, as Christine Scott-Hayward (CSU-Long Beach--criminology) et al., note, "society [reasonably] expects privacy." Increasingly, defense attorneys increasing turn to such research findings in their briefs and pleadings to courts.

While entirely descriptive, The Role of Empirical Scholarship in Fourth Amendment Privacy Jurisprudence levers 13 leading empirical papers in the 4th Amendment space and then identifies and dives deeply into the relatively (and absolutely) small number of judicial opinions that cite to the leading empirical work. The authors' main takeaway is one of frustration with the courts' reluctance to engage with empirical legal scholarship relevant to the question of societal expectations of privacy in the search and seizure context. (To be sure, any judicial reluctance to engage with empirical legal research is merely a sub-set of, for better or worse, courts' collective benign neglect of legal scholarship generally.)

In the end, the paper's findings (and the authors' frustration) both bump into and provide indirect support for a larger point that my Cornell colleague (and friend) Jeff Rachlinski described years ago (2011): “Empirical legal scholarship . . . is not the same as evidence-based law. Empirical legal scholarship resides primarily in the academy.” That relevant empirical legal scholarship will migrate from the academy and into judicial opinions is not a given. Moreover, any such migration will likely continue to vary across legal doctrines and case types.

Taxing Autonomy and Budget Volatility

While complaints from public entities about the policy challenges and harms incident to fiscal uncertainty attributable to budget volatility emerge on an almost annual basis, factors that inform budget volatility remain understudied. Moreover, the particular role that a public entity's taxing autonomy might play remains especially obscure in the empirical literature. However, a recent paper, Does Taxing Authority Help School Districts in Managing Revenue Volatility?, by Laiyang Ke (Ark. St. Univ.--Poli Sci), makes a helpful contribution.

The paper defines taxing autonomy in terms of the degree to which a subnational government (here, public school districts) controls their tax revenues. "Budget volatility" is understood as a function of a school district's "fluctuations and uncertainty in revenue streams and expenditure patterns." Notably, both concepts have particular traction in the US public school district context. First, school districts vary in terms of their taxing autonomy. Second, "previous research has shown that state and local government revenue volatility could lead to expenditure volatility, and uncertain spending fluctuations in school districts may impair education outcomes."

One of the paper's key contributions includes its proposed measure of school district taxing autonomy which draws from school district financial data from the 1998/1999 to 2018/2019 school years. The paper's two-pronged empirical execution features a correlated random effects model as well as D-I-D estimations. The paper's findings emphasize that "school districts with a higher degree of tax autonomy—specifically, those able to change tax rates—experience less current expenditure volatility than districts with lower degrees of tax autonomy." The abstract follows.

"This study examines the impact of tax autonomy on school district expenditure volatility in the U.S., where school districts differ in their ability to levy and adjust taxes, with some having full taxing authority and others relying on fixed appropriations or lacking control over tax rates. This research introduces a comprehensive index to measure taxing autonomy across districts and develops an ordinal classification based on this framework. Using finance data from 1998/1999 to 2018/2019 and difference-in-differences, the study investigates whether districts with higher tax autonomy experience lower expenditure volatility compared to those with limited or no tax authority. The findings reveal that districts with greater tax autonomy, particularly those able to adjust tax rates, exhibit less volatility in current expenditures. Additionally, during the Great Recession, districts with higher autonomy faced smaller cuts in current expenditures by increasing tax revenues, which helped protect educational inputs like pupil-teacher ratios."

RSAs & Chapt. 11

While relatively uncommon prior to the mid-2000s, as Figure 1 (below) illustrates, restructuring support agreements ("RSAs") "have become a regular fixture in Chapter 11 proceedings." RSAs typically lay out a pre-agreed reorganization plan and are usually in place prior to a formal Chapter 11 filing.

Figure 1: RSAs in Cases with Over $100M Assets

While RSAs and their rise in Chapter 11 proceedings remain the subject of sustained debate in normative and theoretical literatures, in a recent paper, Restructuring Support Agreements: An Empirical Analysis, the authors, Katherine Waldock (Columbia), Anthony Casey (Chicago) & Frederick Tung (BU), assess RSAs from an empirical lens. In their effort to "study how RSA contracts evolve over time," the authors lever PACER data archives and focus on more than 400 non-financial Chapter 11 filings that involved more than $100M in declared assets during just over the past two decades. While the analyses are entirely descriptive, key trends are identified in the abstract (below).

"Restructuring support agreements (RSAs), or contracts that commit bankruptcy parties to supporting a plan of reorganization that will conform to certain requirements, are now a common feature of Chapter 11. Parties utilize these agreements in nearly half of all large cases. While prior literature has debated the normative value of RSAs, we take an empirical approach to look at what provisions the parties include in these agreements and how those provisions have changed over time.

Our analysis looks at all RSAs associated with large bankruptcies through the end of 2020. We characterize the types of firms with RSAs, the parties involved, and the language contained within their key sections. A significant majority of RSAs are entered into on or before the petition date. Common provisions relate to the bargaining process during the case and often commit the parties in advance to certain elements of the restructuring plan. Provisions related to control such as debtor milestones have become more frequent in the last ten years of the sample."

Data on Homicide Plea Bargaining in Wisconsin

While various criminal law law theories and issues are increasingly subjected to data, critical wrinkles remain largely "hidden in plain sight." One key issue involves how prosecutors engage in--and deploy--prosecutorial discretion. A deeper and more granular understanding of how prosecutorial discretion cashes out when it comes to plea bargaining remains both pressing and enormously consequential.

Unsurprisingly, data limitations pose one critical barrier to studies on how plea bargaining actually works on the ground. One recent and helpful contribution, Plea Bargaining in Homicide Cases: An Empirical Exploration in One State, involves data on one crime type (homicide), from one state (Wisconsin), between 2005-2018.

Under Wisconsin law, defendants alleged to have committed the most serious homicides can be charged with a Class A felony. Homicide defendants charged with a "lesser" crime, a Class B felony, confront a comparatively lighter sentence. As the authors, Michael O’Hear (Marquette) & Darren Wheelock (Marquette—social & cultural sciences), note, "one common plea deal is for the prosecutor to reduce the charge from a Class A to a Class B homicide in return for the defendant’s surrender of his or her right to a trial." And their paper explores "which Class A defendants get this big break [a Class B felony charge] and why." To do so, the authors examine a set of a 293 cases where the defendant was initially charged with a Class A homicide and then pled guilty. In 198 of these cases, the defendant was permitted to plead guilty to a reduced Class B charge. In the remaining 95 cases, the defendant confronted the original Class A charge (despite having plead out).

Their findings include some possible surprises. One includes the apparent lack of statistical salience of defendant or victim race. Factors that did achieve statistical significance, however, include whether a defendant was defended by a court-appointed lawyer. The paper's abstract follows.

“Even the most serious criminal cases are often resolved through plea bargaining, with potentially dramatic consequences for the sentence ultimately imposed. In Wisconsin, the most serious type of homicide is a Class A felony, which results in a mandatory life sentence. However, many individuals who are initially charged with the Class A offense are given the opportunity to plead guilty to a lesser Class B offense, which does not trigger any mandatory minimum sentence at all. In this Article, we compare the characteristics of the cases in which the defendant pleads guilty to a Class A offense (a relatively harsh outcome) with the characteristics of the cases in which the defendant pleads guilty to a Class B offense (a relatively lenient outcome). We find statistically significant differences between these two sets of cases in such areas as county of prosecution, appointed versus private-pay defense lawyer, and gender of defendant, victim, prosecutor, and defense lawyer.”

From PACER to SCALES: Access to Federal Court Data

Even basic research questions on Art.III courts and federal litigation typically confront consequential barriers to needed data (e.g., federal court records). While the federal government's PACER platform and a few third-party operations provide some relief, both possess their own respective shortcomings and limitations (including, but not limited to, cost).

Initial fruits of a new collaborative effort based at Northwestern University, SCALES, have emerged recently. SCALES is an online platform that seeks to "assemble federal court records, systematically organize them and extract key information, and—most importantly—make them freely available to the public." What exists now includes access to information and records for all federal cases initiated in 2016 and 2017. As SCALES' mission includes "to make court records freely accessible to the public," the platform aspires to, in the future, expand coverage to include "all years."

In a recent paper, The SCALES Project: Making Federal Court Records Free, the authors, David Schwartz (Northwestern) et al., explain the SCALES initiative in greater detail. As well, the project also benefits from its own website (click here).

Initial Data on Trump Judges' Performance

Persistent general public complaints about Trump's Art.III judicial picks include that they are "unqualified" and were drawn from an unduly narrow pool of traditionally plausible candidates. Of course, and as they say, "where data are scant, 'opinions' bloom." In a recent recent paper, How Different Are the Trump Judges, Stephen Choi (NYU) & Mitu Gulati (UVa) bring some initial data to these prevailing conventional wisdoms. The paper sets out to examine how Trump-appointed judges "have done compared to other presidents' appointments." Their findings challenge prevailing and widely-held assumptions.

The paper includes all active circuit court judges between January 1, 2020, through June 30, 2023. While the analyses are largely descriptive, the paper explores judge performance in terms of three key metrics: productivity, influence, and the degree to which a judge is willing to "deviate" from other judges, and particularly from "those closest to them in terms of political ideology." As the paper's abstract (below) implies, and contrary to public predictions of underperformance, the authors find that "Trump judges outperform other judges."

"Donald J. Trump’s presidency broke the mold in many ways, including how to think about judicial appointments. Unlike other recent presidents, Trump was open about how 'his' judges could be depended on to rule in particular ways on key issues important to voters he was courting (e.g., on issues such as guns, religion, and abortion). Other factors such as age and personal loyalty to Trump seemed important criteria. With selection criteria such as these, one might expect that Trump would select from a smaller pool of candidates than other presidents. Given the smaller pool and deviation from traditional norms of picking 'good' judges, we were curious about how the Trump judges performed on a basic set of measures of judging. One prediction is that Trumpian constraints on judicial selection produced a different set of judges that underperform compared to judges appointed by other presidents. Using data on active federal appeals court judges from January 1, 2020 to June 30, 2023, we examine data on judges across three different measures: opinion production, influence (measured by citations), and independence or what we refer to as “maverick” behavior. Contrary to the prediction of underperformance, Trump judges outperform other judges, with the very top rankings of judges predominantly filled by Trump judges."

Judicial Decisionmaking "In the Time of COVID-19"

As the scope and reach of the COVID-19 pandemic's impact remain difficult to over-state, it will surprise few to learn that the full reach of public health officials' legal/statutory claims did not escape judicial scrutiny. What is not yet well understood, however, is a more granular understanding of how legal decisions informed particular legal doctrines, including those involving religious liberty and statutory interpretation in the public health silo. To this end, a recent paper by Michelle Mello et al., Judicial Decisions Constraining Public Health Powers During COVID-19: Implications for Public Health Policy Making, takes an initial empirical cut.

Deploying "legal epidemiology," the paper analyzes 112 federal and state courts decisions between March 2020 and March 2023 that involve successful plaintiff challenges to federal, state, or local pandemic-motivated "mitigation" orders. The authors argue that the 112 cases reveal "areas of profound instability in how courts analyze challenges to exercises of public health legal powers." The disruptions were particularly acute in the religious liberty and statutory authority areas. The findings make clear that going forward assertions of public health authority will unlikely benefit from "broad [judicial] deference." The paper's abstract follows.

"Public health legal powers are increasingly under pressure from the courts in the United States. During the COVID-19 pandemic, individuals and organizations successfully challenged many community mitigation orders (for example, mask mandates, vaccination mandates, and restrictions on gatherings), demonstrating the legal vulnerability of disease control measures. Analyzing 112 judicial decisions in which the plaintiff prevailed from March 2020 through March 2023, we examined the ways in which courts constrained public health powers during the COVID-19 pandemic. We found that in these 112 decisions, courts shifted how they analyze religious liberty claims and reviewed challenges to the exercise of statutory powers by health officials in novel ways. We discuss implications for public health policy going forward, and we recommend ways in which legislatures and health officials can design policies to maximize their prospects of surviving legal challenges."

Municipal Revenue Generation and Enforcement Incentives

While an emerging literature explores possible links between government revenue generation and enforcement incentives, much of the literature is theoretical. Even though conflicts of interest are rife in this context, empirical testing is comparatively scarce.

In a recent paper, Agency Incentives and Disparate Revenue Collection: Evidence from Chicago Parking Tickets, Elizabeth Luh (SEC) et al. explore whether a recent increase in motor vehicle registration fees as well as related fines for non-compliance in Chicago informed the Chicago Police Department's ("CPD") parking enforcement patterns. By increasing both the cost of motor vehicle registration and the fine for non-compliance in 2012, the City of Chicago simultaneously made vehicle registration compliance more expensive and enforcement more profitable relative to other parking fines.

The paper executes a difference-in-differences (DiD) strategy to assess the relative change in sticker enforcement across various types of neighborhoods, "focusing on Black versus non-Black areas due to the purported claims of disparate impacts from the general public." The paper reports that CPD vehicle registration enforcement increased by nearly 2,500 tickets in Black relative to non-Black neighborhoods per year. It also finds that "non-sticker ticket enforcement significantly increased by nearly 12,000 tickets, despite the non-sticker fines remaining largely the same." The authors interpret these patterns as evidence consistent with a broader revenue collection effort. The paper's abstract follows.

"We leverage a sharp 2012 parking fine increase for failing to purchase vehicle registration to examine disparate ticketing patterns across enforcement agencies in Chicago. Using an event-study framework, we find that Chicago police increased their enforcement of car registration non-compliance in Black relative to non-Black neighborhoods, with no observed disparate response for non-police enforcement agencies. This disparity is unexplained by differences in non-compliance and is instead driven by departmental revenue incentives and lower marginal search costs in Black neighborhoods. Disparate enforcement also exacerbated existing gaps in financial instability, including increased rates of ticket non-payment and bankruptcy filings in Black neighborhoods."

Farmland Property Transfers Across Generations in the Mid-20th Century

Farmland property succession is a central issue in agricultural policy, and succession in the agricultural sector is quickly becoming even more salient owing to the number of farmland transfers expected in the coming years. Nevertheless, and owing to a paucity of data, the scarcity of research on farmland property transfers across generations is, according to Adrian Haws (Cornell-Econ.) et al., “jarring.”

In their recent paper, Who (Actually) Gets the Farm? Intergenerational Farm Succession in the United States, the authors link Census microdata for census records for 1.3 million farmers in 1900 or 1910 to their children’s succession outcomes up to 40 years later. Such a data set provides glimpses into which farmers’ children, if any, took title to farmland property.

A within-family identification strategy yielded some notable findings, including that daughters were rarely successors and evidence of a small and monotonic birth order effect among sons. An excerpted abstract follows.

“Farm succession is a central issue in agricultural policy. Yet while many studies explore succession planning, little is known about how farms are actually transferred. … Using a within-family identification strategy, we find that first-born sons are slightly more likely than their younger brothers to be successors while their parents are working aged. However, birth order is not predictive of who receives the farm when parents are older or deceased. For later farm transfers, sons who were previously tenant farmers are much more likely than their brothers to be successors, possibly because they are better prepared. Fewer than one-fifth of farmers transfer their farm to any son….”

Birth Order Effects in Developing Countries

While birth order effect studies' popularity has waned a bit over time, most focus on developed countries. The consistently negative results in developed countries imply that "… the later a child is born within a family, the worse their adult economic outcomes relative to their earlier- born siblings” incident to potential intra-familial (small- or capital D) discrimination. By contrast, the comparatively smaller number of studies that use data from developing countries yield unstable results.

A recent paper, Birth Order Effects on Education: Insights from Low- and Middle-Income Countries, sets out to fill a scholarly void involving involving emerging nations. In it, Silvia Mendolia (Univ. Turin) et al., estimate mothers’ fixed effects models assessing the impact of birth order on schooling and school attendance. The models exploit "almost 1.8 million observations gathered from the Demographic and Health Survey (DHS) to measure birth order effects on children’s educational outcomes in 35 developing countries, between the mid-1980s and 2020."

Overall, the paper finds evidence of negative birth order effects on educational attainment in 32 out of 35 developing countries. Notably, the paper finds that in most developing countries, “overall birth order effects do not vary by gender, family wealth, location or over time.” The paper's (excerpted) abstract follows.

"... To the best of our knowledge, this is the first study analyzing birth order effects in a comprehensive set of developing countries. In developing countries, families tend to be relatively large and within-family resources scarce. The DHS contains harmonized data and variables for all countries, providing a picture of birth order effects that is consistent across the developing world. Using mothers’ fixed effects models, we estimate the impact of birth order on standardized years of schooling and school attendance, exploring non-linearities in birth order effects, as well as heterogeneous effects by gender, socio-economic characteristics and over time. Overall, we find negative birth order effects on educational attainment in 32 out of 35 countries. Consistent with this, we find that the probability of not being in school increases with birth order. We find that in most countries, overall birth order effects do not vary by gender, family wealth, location or over time...."

Data on Police Separations

While national news and policymakers focused on a surge in police separations in many large city departments and identified social unrest incident to the George Floyd murder as a likely contributor, much of the public discussion unfolded amid a paucity of data. By contrast, a recent paper by Ben Grunwald (Duke), A Large-Scale Study of the Police Retention Crisis, sets out to submit prevailing conventional wisdoms to data.

To do so, the paper exploits data drawn from 6,800 local law enforcement agencies across 15 states that, taken together, serve one-half of the U.S. population. As the Figure (below) descriptively implies, the acceleration in police officer separations beginning in the summer of 2020 was not off-set by any increasing in police hiring.

While what to infer from these two trend lines is not obvious, the paper lands on two main descriptive findings. First, "the increase in separations in IPED agencies after the summer of 2020 was smaller, later, less sudden, and possibly less pervasive than the retention-crisis narrative suggests. All told, the cumulative impact on the labor force by the end of 2021 was just 1%." Second, these "[a]ggregate figures, however, mask variation at the agency level. As I show, a substantial minority of large agencies meaningfully shrank by the end of 2021.”

JLEA

Further evidence of a healthy, robust, and growing scholarly field is provided by the emergence of an important new online journal in the ELS space. Co-edited by Christoph Engel (Max Plank Institute), Lee Epstein (USC), Daniel Klerman (USC), and Eyal Zamir (Hebrew) and published by Sage, the Journal of Law & Empirical Analysis (JLEA), a peer-reviewed, open access journal, "aspires to spearhead the empirical turn in law by publishing cutting-edge research that is accessible and of interest to the wider legal community. JLEA aims to attract papers written by social scientists and legal academics alike that have clear legal relevance, that appeal to an international audience, and that combine theory and empirics, whether to draw inferences or to offer rich description."

Information on its inaugural issue can be found here.

Federal Court Judicial Opinons and Judge Gender

For years (decades, even), legal and political science literatures have considered whether judge gender informs judicial outcomes. While the standard interpretation emphasizes that judicial attributes (including judge gender) play a causal role, an alternative explanation considers the possibility that case characteristics (and therefore potentially outcome) become associated with judicial attributes via some other mechanism, such as case assignment, or decisions about what cases to publish or report.

Results in a recent paper, Gendered Judicial Opinions, by Michael Livermore (UVa) et al., substantially support an assignment/selection pathway explanation. To be sure, the paper is appropriately cautious cautious and notes that while its findings support the assignment/selection pathway, the possibility of a gender causation pathway cannot be properly excluded as the two are not mutually exclusive.

The paper levers a rich data set that includes all federal circuit court opinions, published and unpublished, from 2001 through 2017, except for decisions of the Court of Appeals for the Federal Circuit, en banc opinions, and opinions that include fewer than 900 characters. The authors then merged textual data from judicial opinions with judge-level demographic information collected by the Federal Judicial Center (FJC). The paper's abstract follows.

“In this paper we investigate whether gender is associated with the content of judicial opinions in the U.S. courts of appeals. Using a topic model analysis, we find that gender is a significant predictor of the content of judicial opinions. Two causal pathways could explain this result: (1) men and women judges write differently about the same cases; (2) men and women judges write about different cases, either due to assignment or selection effects. To untangle these two pathways, we carry out three additional analyses. First, we examine whether the United States as a party is associated with judge gender. We next examine whether case codes are associated with judge gender. Finally, we examine the relationship between topic prevalence and gender, controlling for case codes. Our findings lend greater support to the second pathway than the first. This result raises the prospect that prior work on gender-based differences in judicial behavior may be confounded by assignment or selection effects. Our results also raise normative concerns about gender disparities in voice and influence in the U.S. courts.”

Regulation & 3rd-Party Litigation Financing

As the world of 3rd-party litigation financing ("LTPF") grows, state efforts to regulate this industry sector endeavor to keep pace. While much of this activity takes place outside of public view, scholarly attention is increasing.

One example includes a recent paper by Ronen Avraham (Tel Aviv, Texas), Anthony Sebok (Cardozo) & Joanna Shepherd (Emory), The Whac-A-Mole Game: An Empirical Analysis of the Regulation of Litigant Third Party Financing. What the paper finds, unsurprisingly, is that as states ramp up regulation on LTPF, funding activity declines; where loosening of state regulations corresponds with increased funding activity.

Another core finding, and inspiration for the paper's title, is that funders continually adjust contract terms in an effort to circumvent regulations. To illustrate, "when legislation prohibits compounding or limits how long funders can charge interest, the funder responds by increasing the posted monthly interest rate. Although these interest rate increases are not enough to completely offset the impact of the other restrictions on the funder’s returns, the funder nevertheless has higher returns than it would have without the attempted circumvention." Thus, akin to the "classic Whac-A-Mole game, funders try to maintain their pre-regulation per-funding-return no matter how states’ regulatory activity tries to get rid of it."

The Empirics of Rental Evictions in Kentucky

As statutory eviction pauses implemented during the COVID-19 pandemic expire, eviction rates are beginning to rebound to pre-pandemic levels. Though this issue is fraught with an array of legal and policy implications, the topic is also noted for a paucity of empirical research. A recent paper, Rural Renting: An Empirical Portrait of Eviction, by Cassie Chambers Armstrong (Louisville) & Christopher J. Ryan, Jr. (IU-Bloomington), makes a helpful contribution by focusing on rental eviction outcomes in Kentucky. Specifically, the paper analyzes factors "influencing the eviction process in the rental market—and the likelihood that it will result in a judgment against a renter— once a property owner initiates legal proceedings."

To this end, the paper levers data gathered by the Kentucky Administrative Office of the Courts (“KOC”) from all 120 Kentucky counties from 2018-2022, yielding usable data on 202,572 renter evictions. One data set limitation is that it excludes information on pre-filing interactions and negotiations between renters and property owners.

Key findings include that those living in rural areas were more than 55 percent more likely to experience a judgment of eviction just by virtue of where they lived and those living in jurisdictions that had adopted renter-friendly policies (URLTA) were more likely to avoid a judgment of eviction—even though these laws do not impact the eviction process itself. Interestingly, renters who lived in the same ZIP code and/or state as their landlord were significantly more likely to resolve their case prior to judgment. An excerpted abstract follows.

"... Collectively, these findings, and others, have implications for eviction policy. They suggest that we must pay more attention to rural courts and the factors that lead to disparate outcomes in them. They call for programs that encourage collaboration between property owners and home renters, such as mediation and other types of eviction diversion. These results also must lead to further study of the time period between when an eviction case is filed and when it is concluded to more fully understand how the various parties approach decision-making with this legal process...."

Initial Data on Data Breach Litigation

News reports about massive data breaches are quickly becoming all-too common. While states have adopted data breach notification statutes and an increasing number of privacy statutes, litigation in this space remains largely by common law doctrines. Despite steady increases in litigation over time, quite little is know, systematically, about how such litigation has unfolded.

At the risk of stacking "turtles all the way down," a recent paper, Unto the (Data) Breach, sets out to gather initial data on data breach litigation. Drawing from WestLaw and PACER documents, the paper identifies 225 cases (between 2005 and 2022). While the analyses are exclusively descriptive, this is representative of helpful granular foundational work from which more technically sophisticated analyses can flow in the future. One take-away from the paper is that a comparative absence of a "cohesive data breach common law" likely exerts upward pressure on settlement activity. An excerpted abstract follows.

“Since the early 2000s, U.S. courts have begun hearing “data breach” liability cases, the inevitable result of a growing internet-connected technology infrastructure. The relatively recent development of case law signals a body of law in development, stunted by significant limiting factors that prevent the coalescence of legal principles. … This descriptive empirical study analyzes, in detail, 225 data breach cases from 2005-2022, reviewing these cases over a three-year period to descriptively identify key trends and changes within a case’s life on the docket.

This study identifies the type of plaintiff, settlement amounts, type and disposition of information compromised, claims, common motions, and key strategies most likely to result in a favorable plaintiff outcome. It also explores broad trends in data breach litigation, including acceptance of claims by courts, the status of future harms in 12(b)(1) and 12(b)(6) standing challenges, and the degree to which courts are willing to let cases proceed beyond preliminary motions….”

Data on U.S. Solicitor General's "Flips"

While many criticize the Supreme Court for "unsettling" Court precedent, exceptionally few focus on instances where the Solicitor General's Office ("SG") "advances legal arguments that differ from those advanced by government lawyers in previous cases." Notably, '“[e]ven those observers who defend the SG, including veterans of the office, caution that inconsistency in legal argument poses a threat to the SG’s credibility with the Court." A recent paper by Margaret Lemos (Duke) & Deborah Widiss (Indiana), The Solicitor General, Consistency, and Credibility, sets out to empirically better understand the circumstances that lead the SG to "change its position on the meaning of the law, and to unpack the connections between consistency and credibility.”

To do so, the paper leverages an original dataset that includes 131 cases drawn from 1892 to the Court's 2022 term. The authors make clear that the sample is under-inclusive in that it contains only those SG "flips" that are "evident": that is, where "the government itself, other parties or amici, or the justices flagged that the government had changed its legal position." In about 80% of the cases, the SG itself acknowledged the change in its briefing. Of course, that also means that in 20% of the cases the government "either did not discuss, or denied, the changed position."

While the paper's analyses are strictly descriptive, main findings include that changes in the SG's litigating positions have increased over time. The authors also emphasize that SG reversals (or "flips") happen for an array of (often overlapping) reasons, "many of which stem from the SG’s unique role in coordinating litigation across a vast and constantly changing federal government." An excerpted abstract follows.

“This Article offers the first comprehensive look at cases in which the Solicitor General (SG) rejects a legal argument offered on behalf of the United States in prior litigation. Such reversals have received considerable attention in recent years, as shifts in presidential administrations have produced multiple high-profile “flip-flops”—as the justices sometimes call them—by the SG. Even those observers who defend the SG, including veterans of the office, caution that inconsistency in legal argument poses a threat to the SG’s credibility with the Court. Our goal is to better understand the circumstances that lead the SG to change its position on the meaning of the law, and to unpack the connections between consistency and credibility.

… Indeed, our study calls into question the idea that ideological swings associated with changes of presidential administrations can be isolated, either in theory or in practice, from other sorts of legal, social, and technological changes that shape the government’s understanding of the law. It also shows that the connection between consistency and credibility, while intuitive at first blush, rests on a formalist understanding of law and an unpersuasive equation of the judiciary and the executive.”

COVID-19 Relief Funding: Following the Money Across States

In response to the global COVID-19 pandemic, the U.S. federal government, through the CARES Act, FFCRA, RRA, and ARPA, provided as much as "$6 trillion in income support to households, a combination of loans, grants, and tax relief to firms and non-profits, funding for (public) health efforts, and intragovernmental grants to sub-national governments." These federal funds included "roughly $900 billion in aid to state and local government entities." And a recent paper, What Do State Do With Fiscal Windfalls? Evidence from the Pandemic, by Jeffrey Clemens (UCSD--econ.) et al., focuses on how state governments deployed this federal aid.

To do so, the paper benefits from a unique dataset constructed from a rich array of sources and an estimation strategy that levers variations across states with "low vs. high levels of per resident representation in the U.S. Congress." More specifically, owing to the possibility that federal aid to states may be endogenous along many dimensions, the paper deploys "an instrumental-variable approach to estimate the effect of federal relief to state and local governments on the revenue and expenditure decisions of state governments."

Among the findings include that "for every $1000 of committed federal aid per capita, $645 appears as increased intergovernmental grant revenue in states over the years 2020-2022." When it came to deploying the federal funds, state governments emphasized "General Government Expenditures ($379 per $1000 of federal aid) and state and local pension contributions ($72 per $1000 of federal aid)." An excerpted abstract follows.

“Using variation in federal pandemic-era fiscal aid to states driven by the strength of political representation, we find that incremental pandemic-era fiscal aid to states was most likely to end up in the categories of general administrative service spending and employee pension benefit funding. Spending on categories that motivated the aid in the first place, such as healthcare, education, and infrastructure, may also have responded but does not show robust patterns. Total state government revenues and expenditures had increased by around 70 cents per incremental windfall dollar of committed federal funds by 2022. Of this, the statistically significant categorical spending effects are 38 cents to general government expenditures (the residual that in principle excludes healthcare, education, infrastructure, and other functional categories) and 7 cents to pension funding, even though the latter use was inconsistent with the objectives of the legislation.”

"Too-Big-To-Fail" Banks, Bailouts, and Fairness

While such questions as "are regulators across countries 'in the pockets' of large banks?" and "do supervisors provide favorable treatment when bailing out large, global systemically important banks?" persist, conventional wisdom strongly implies that "regulators around the world give [Too-Big-To-Fail] TBTF banks preferential treatment in bailouts relative to other banks, often viewed as an unfair advantage for these banks."

While an unfair regulatory advantage tilting in favor of TBTF banks has "been engrained in the minds of banking researchers, policy makers, and the public around the world for decades," to date little empirical work has subject this belief to data. In a recent paper, Do “Too-Big-To-Fail” Banks Receive Preferential Treatment in Bailouts? Surprising Results from a Cross-Country Analysis, Allen Berger (Univ. S. Car.—Business) et al. do just that and find evidence that "soundly reject(s) implications of unfair preferential treatment concept."

The authors lever hand-collected bank bailout data in the EU from 2008-2014. Their sample includes 100 banks whose "cumulated total assets represent about 50 percent of the total assets of EU banks at the end of 2014" and "35 bailout events for capital injections and 61 bailout events for debt guarantees. Among these, 10 capital injections and 24 debt guarantee bailouts are provided to large, systemically important banks.”

The paper's findings largely contradict prevailing conventional wisdom. "We find statistically and economically significant evidence that regulators treat [Globally-Systemically Important Banks] G-SIBs more harshly than other banks. Regulators make G-SIBs wait longer for bailouts until their capital ratios have more significantly deteriorated, apply more severe restrictions to them, and withdraw government bailout support sooner at lower capital ratios than for other banks in their countries.” The abstract follows.

"Regulators more often bail out “Too-Big-To-Fail” banks than others, but this may not imply preferential treatment as commonly believed. Bailouts are complex dynamic processes involving more than one-time aid, so harsh treatments elsewhere in the process may counter the benefits of the higher likelihood of bailouts for these banks. Using bailout data from 22 European countries we find relatively harsh treatment for Globally-Systemically Important Banks. Regulators bail out G-SIBs at later stages of financial deterioration, impose stronger restrictions, and withdraw aid after less significant recoveries. We explain these findings using cross-country data on supervisory powers, political connections, and national culture."

Circuit Panel Effects Over Time

Most empirical judicial decisionmaking scholarship on federal circuit courts tries to exploit something close to quasi-random assignments to three-judge appellate panels. While attention typically focuses on judge votes, in a recent paper, Partisan Panel Composition and Reliance on Earlier Opinions in the Circuit Courts, Stuart Minor Benjamin (Duke) et al. instead focus on whether panel composition (partisanship) influences how judges rely on prior case law.

Drawing on a data set that includes "all published and unpublished federal appellate opinions in the Lexis database that were issued between 1974 and 2017," the authors find that circuit panel composition does influence which cases are relied upon and followed and that the influence is asymmetric. Specifically, they find that evidence of a partisan influence in this context is: 1) relatively recent; 2) most salient for circuit opinions written by 3-Democrat panels; and 3) more likely in relatively "high-profile" cases. An excerpted abstract follows.

"Does the partisan composition of three-judge panels affect how earlier opinions are treated and thus how the law develops? Using a novel data set of Shepard’s treatments for all cases decided in the U.S. courts of appeals from 1974 to 2017, we investigate three different versions of this question. … We find that partisanship does, in fact, structure whether earlier opinions are followed and that these partisan effects have grown over time—particularly within the subset of cases that we believe are most likely to be ideologically salient. Since legal doctrine is developed by building upon or diminishing past opinions, these results have important implications for our understanding of the development of the law."

"Red States," "Blue States," and State Criminal Codes

The political science literature increasingly paints a picture of distinctive "red" and "blue" states that separate from each other on a growing number of issues. If accurate, one may plausibly assume that this separation would increasingly be reflected in differences among "red" and "blue" state criminal codes, especially because, as Paul Robinson (Penn.) et al. argue, "a jurisdiction’s criminal law rules commonly reflect that community’s most basic shared values, probably more than many if not most other areas of legislation."

In a recent paper, Red Codes, Blue Codes? Factors Influencing the Formation of Criminal Law Rules, Robinson et al., test this assumption by comparing state criminal codes from "the six largest deep blue (Democratic) states – California, Colorado, New York, Illinois, New Jersey, and Washington – and the six largest deep red (Republican) states – Texas, Florida, Ohio, Georgia, Tennessee, and Missouri – in which the same party has controlled the governorship and both houses of the legislature for at least the last three elections, and often much longer." The paper's set-up itself is important as the authors self-consciously "stack the deck" toward  finding state criminal code differences.

Despite an ex ante assumption that deeply red and deeply blue criminal law rules would differ and a comparison that tilts toward finding differences, the authors find "that the red-blue effect seems to appear only in those few instances in which a criminal law rule becomes a public political issue. For the wide variety of issues for which one might expect a strong red-blue difference, our research shows no such clear difference. An excerpted (heavily) abstract follows.

"In other words, not only is the red-blue divide of little effect for the vast bulk of criminal law, but the factors that do have effect are numerous and varied. The U.S. does not in fact have red codes and blue codes. More importantly, the dynamics of criminal law formulation suggest that distinctive red codes and blue codes are never likely to exist because the formulation of most criminal law rules are the product of a complex collection of influences apart from red-blue."

Patents, the Mayo Decision (2012) & Innovation

Our patent system seeks to promote innovation. When it comes to levering incentives in the service of innovation, property rights flowing from patent grants do much of the expected work.

The Supreme Court's 2012 Mayo v. Prometheus decision disrupted the existing status quo and articulated a new legal test for patent-eligible subject matter. Assessing Mayo's impact in general and on molecular diagnostics innovation in particular confronts considerable data and research design challenges.

Arti Rai (Duke) et al. take up this challenge in a recent paper, Molecular Diagnostics Patenting after Mayo v. Prometheus: An Empirical Analysis. As the authors make clear, "in an ideal world, we would be able to examine rigorously differences in molecular diagnostics innovation before and after Mayo. Unfortunately, the rigorous measurement of innovation raises well-known challenges.” Despite these (and other) challenges, the paper uses data on molecular diagnostic patenting from 2009 through 2019. The paper's DID specifications exploit comparisons with non-bioinformatics and non-molecular diagnostic biochemistry inventions. The paper reports mixed results summarized in the excerpted abstract that follows.

"This Article contributes to the presently thin evidence base on the impact of Mayo by using data on patent applications, examinations, and grants from 2010-2019 to comprehensively trace the effects of the test and subsequent related developments. Using descriptive data as well as a difference-in-difference (DID) design, we evaluate the extent to which the decision was followed by one of three expected outcomes: a decline in patent quantity and quality (“retrenchment”); increase in patent prosecution “toughness”; and applicant “adaptation.” We find substantial support for our toughness and adaptation hypotheses, but not our retrenchment hypothesis: molecular diagnostic patenting did not decline in aggregate, though there is some evidence of a relative decline among small, U.S. based firms as well as a potential decline in the share of patents covering new (non-continuation) inventions. These results suggest that molecular diagnostic patents are harder to get but they are still being applied for and granted, with their narrowed scope making them less likely to block follow on innovation.”

The Bruen Decision After One Year: An Empirical Look

In its 2022 Bruen decision, the Court ruled (6-3) that a New York law requiring that applicants “show cause” in their conceal carry firearm license applications unconstitutionally infringed on Second Amendment protections.

Since the Bruen decision, a stream of additional lawsuits challenging various other state and federal firearm regulations has been decided. Assessing these post- Bruen court decisions may hold some clues about the Bruen decision’s impact on the Second Amendment landscape.

To this end, in One Year Post-Bruen: An Empirical Assessment, Eric Ruben (SMU) et al., analyze more than 450 claims, flowing from 326 cases, decided after Bruen. The authors note that the pace (and volume) of litigation dramatically increased after Bruen as well as the lawsuits’ efficacy. The authors also observe that Bruen’s analytic framework “has not meaningfully constrained judges.” The paper’s abstract follows.

"In the year after New York State Rifle & Pistol Association v. Bruen, a steady stream of highly publicized opinions struck down a wide range of previously upheld gun restrictions. Courts declared unconstitutional policies ranging from assault weapon bans to domestic abuser prohibitions to various limits on publicly carrying handguns. Those opinions can frequently be paired with others reaching the opposite conclusion. The extent to which Bruen shook up the Second Amendment landscape and has caused widespread confusion in the courts is starting to come into focus.

This Article measures Bruen’s after effects by statistically analyzing a year’s worth of Second Amendment opinions. We coded more than 450 challenges for dozens of variables including both case and judge characteristics, resulting in a comprehensive post-Bruen Second Amendment dataset. The findings of our analysis provide an objective basis for assessing the upheaval wrought by Bruen and highlight both unanswered questions and immense challenges for Second Amendment doctrine in the coming years."

New Thinking About an Old Problem: Selection Effects

Keith Hylton (Boston Univ.) and Sanghoon Kim (Buffalo--econ.) distill their recent paper's challenge nicely. When it comes to much of empirical work on legal (and court) outcomes, the full potential of multiple regression "has not been realized, largely because of one reason: the information contained in court opinions comes from a selected sample."

For all the obvious reasons, much of empirical legal research exploits court outcomes as data. One challenge, however, is that the comparatively smaller subset of formal legal disputes that persist to a judicial outcome almost assuredly systematically differs from the much larger subset of legal disputes that privately settle. My former mentor and colleague, Ted Eisenberg, frequently reminded me that "one cannot over-state the importance of selection effects in empirical legal research."

While the challenge endures, solutions are not entirely obvious. In Trial Selection and Estimating Damages Equations, Hylton and Kim develop an interesting suggestion. Specifically, they articulate a structural econometric model that, they argue, better accounts for litigant selections in the appellate context. The paper's abstract follows.

"Many studies have employed regression analysis with data drawn from court opinions. For example, an analyst might use regression analysis to determine the factors that explain the size of damages awards or the factors that determine the probability that the plaintiff will prevail at trial or on appeal. However, the full potential of multiple regression analysis in legal research has not been realized, largely because of the sample selection problem. We propose a method for controlling for sample selection bias using data from court opinions."

Do Patent Reforms Enhance Innovation?

Over the past two decades, academics and policy makers in the IP/patent space have focused on three specific developments: elimination of presumptive injunctive relief for victorious patent enforcers in eBay Inc., creation of the Patent Trial and Appeal Board (PTAB), and restriction of software’s eligibility for patent protection in the Alice Corp. decision. Despite general agreement on the importance of these three patent system changes, their individual and collective impact on innovation (construed in terms of firms’ patent filings and R&D investments) remains both contested and understudied.

A recent paper by Christian Helmers (Santa Clara—business) and Brian Love (Santa Clara), Patent Law Reform and Innovation: An Empirical Assessment of the Last 20 Years, jumps into this empirical void and seeks to assess these three reforms’ impact on innovation in the U.S. To do so, the paper constructs firm-level innovation measures and of exposure to each reform and submits them to various Dif-in-Dif specifications that compare relatively exposed and relatively unexposed firms. The paper finds that “Overall, our results suggest that these three reforms had a positive effect on innovation as measured by patent filings and R&D investment.” More detailed findings are described in the excerpted abstract.

“... We find: a positive association between eBay and R&D spending by firms that were relatively more exposed to patent litigation prior to the Court’s decision; a positive association between the introduction of PTAB proceedings and both R&D expenditures and patent filings by firms that innovate in tech classes where PTAB has been most active; and a positive association between Alice and both R&D spending by software firms and patenting by firms that held relatively more software patents prior to the Court’s opinion.”

Deploying Large Language Models in the Service of Empirical Legal Studies

The release of ChatGPT, among other large language models (LLMs), materially altered the natural language processessing landscape. Questions about whether and, more to the point, how, these tools could advance empirical legal studies quickly emerged and, unsurprisingly, persist.

In a recent paper, Language Model Interpretability and Empirical Legal Studies, Michael Livermore (UVa), et al., undertake two tasks: "(1) to present an empirical comparison of LLMs and non-LLM on a sampling of fundamental tasks, and (2) to examine the use of LLMs within the norms of ELS scholarship." The paper specifically seeks to place "the current capabilities and limitations of LLMs within the larger landscape of techniques used to convert law into data." The authors report that their comparisons yielded "mixed" results, noting that existing LLMs outperform non-LLM natural language processessing techniques in some contexts but not in others. The paper's abstract follows.

"Large language models (LLMs) now perform extremely well on many natural language processing tasks. Their ability to convert legal texts to data may offer empirical legal studies (ELS) scholars a low-cost alternative to research assistants in many contexts. However, less complex computational language models, such as topic modeling and sentiment analysis, are more interpretable than LLMs. In this paper we highlight these differences by comparing LLMs with less complex models on three ELS-related tasks. Our findings suggest that ELS research will—for the time being—benefit from combining LLMs with other techniques to optimize the strengths of each approach."

Federal Judicial Data: Greater Transparency or Opacity?

While not itself an empirical paper per se, The Necessary and Proper Stewardship of Judicial Data, by Zachary Clopton (Northwestern) and Aziz Huq (Chicago), engages in a discussion near-and-dear to most legal empiricists' heart. Specifically, their paper sets out to develop "the first comprehensive descriptive, doctrinal, and normative analysis of judicial data.”

As the authors note, judicial (especially Art. III courts) data are "valuable in governance and for those interested in understanding how they are being governed. In the absence of hurdles or walls, it can illuminate the operation of one branch of the national government. Constitutional lapses can be brought to light through retail or wholesale data." As such, data flowing from Art. III courts and institutions can promote various public goods by, in part, advancing a better understanding of, or improvements to, judicial institutions.

One critical anomaly is that, as compared to the legislative and Executive branches, judicial branch data remain "largely terra incognito for scholars and the general public alike. Instead, public attention rushes to high-profile cases of leaked Supreme Court opinions and private influence campaigns.” While why this is so remains unclear (to me, anyway), that it is so, however, is obvious to any empirical legal scholar who labors and writes in this space. And the stakes incident to the "transparency v. opacity" debate grow more acute as new scholarly tools emerge (e.g., natural language processessing, text-scrapping).

As the authors make clear, under current policy and practice judicial data "might be lost, dissipating into air, or kept under wraps, never yielding up their secrets." In contrast, a commitment to greater transparency invites characterizing judicial data as a "new public asset that can then be tapped to create public goods for the benefit of all."

School and Crime; Cause and Effect

Criminologists increasingly worry about a "school-to-prison pipeline" problem. Such worries are typically inflamed incident to public school districts' typical reaction (or, to some, over-reaction) to tragic episodes of student violence, especially mass-violence. An increasing number of empirical efforts seek to identify and better understand causal mechanisms. To this end, a recent paper by Todd Jones (Miss. St.-econ) and Ezra Karger (Fed. Res.-Chicago), School and Crime, exploits data from the National Crime Victimization Survey and explores "seasonality" patterns.

As it relates to "seasonality" patterns, the paper's main take-away is that juvenile criminal activity in general, and student-on-student crime in particular, largely maps onto the school calendar. More specifically, the paper documents a population-level outcome: across all children—those who are compliers for education-based policies and those who are not—attending school causes an average increase in the likelihood of student-on-student crime. While the overall patter is suggestive, the underlying mechanisms are less so. That is, whether a school's institutional setting itself (e.g., institutional-level policies and practices)--or, for example, that student-on-student crime is simply a function of student density levels--is the critical mechanism remains unclear. The paper's abstract follows.

“Criminal activity is seasonal, peaking in the summer and declining through the winter. We provide the first evidence that arrests of children and reported crimes involving children follow a different pattern: peaking during the school year and declining in the summer. We use a regression discontinuity design surrounding the exact start and end dates of the school year to show that this pattern is caused by school: children aged 10–17 are roughly 50% more likely to be involved in a reported crime during the beginning of the school year relative to the weeks before school begins. This sharp increase is driven by student-on-student crimes occurring in school and during school hours. We use the timing of these patterns and a seasonal adjustment to argue that school increases reported crime rates (and arrests) involving 10–17-year-old offenders by 47% (41%) annually relative to a counterfactual where crime rates follow typical seasonal patterns. School exacerbates preexisting sex-based and race-based inequality in reported crime and arrest rates, increasing both the Black-white and male-female gap in reported juvenile crime and arrest rates by more than 40%.”

Legal Precedent and Decay

While the idea of legal precedent implies some degree of durability, precedents are not immortal. Over time, they lose their capacity to influence action; eventually they cease to be of anything other than historical interest. Although precedential decay's existence is generally accepted, the determinants of precedential decay are comparatively under-studied.

Yun-chien Chang (Cornell) & Geoffrey Miller’s (NYU) recent paper, Decay of Precedent in State Supreme Courts, extends this literature by analyzing citation decay patterns in state supreme court opinions, drawing on three different (and large) data bases. Their paper examines both “backward-looking” precedential decay—the frequency with which opinions in a given year cite to opinions from prior years as well as “forward-looking” decay—the frequency with which specific precedents are cited in later opinions. One key finding includes the extraordinary persistence of exponential decay across all of the data. The paper’s abstract follows.

“This paper investigates the decay of precedents in three unique databases: citations in all published opinions by state supreme courts in 2019 (102,555 citations); citations in all published opinions by state supreme courts in 2003 (114,777 citations); and citations by all state and federal courts to state supreme court opinions issued in 2003 (259,627 citations). We find that the frequency of citation drops off by a roughly constant ratio with each passing year. The large scale of our yearly data and the time studied (1904–2019) suggests that this pattern must be the result of underlying forces not having to do with particular circumstances of time or place. We also find that cases with dissents tend to cite older cases and longer decisions tend to be cited more often and faster. Criminal law cases, as compared to constitutional law cases, tend to cite more recent precedents, while trust and estate cases tend to cite older precedents. We offer some conjectures about possible reasons for the observed differences in decay rates.”

How to Value Data

While a recent paper, Data Valuation and Law, is not itself an "empirical paper," it is certainly "ELS-adjacent" and, as such, I thought it might interest some. This is particularly so as both AI and "big data" become increasingly important.

Motivating Jordan Berry (USC) and D. Daniel Sokol's (USC) paper is the observation that "while data has become increasingly valuable and important, the law’s attempts to value data have lagged, remaining confused and underdeveloped." To improve matters in this regard, the paper advances a “real options” approach for framing data valuation not because the approach is perfect but, rather, because “it is the least bad alternative available.” The paper's abstract follows.

"Data has become an increasingly valuable asset. Numerous areas of law—including contracts, corporate law, IP, antitrust, tax, privacy, and bankruptcy—require parties and courts to determine the value of assets, including data. Unfortunately, data valuation has been hindered by a lack of clarity over what data is and why it is valuable. This lack of clarity also increases the chances of legal decisionmakers valuing data in inconsistent ways, which would create further confusion, inefficiencies, and opportunities for regulatory arbitrage.

This article proposes a unified framework for valuing data that will promote consistent valuations across fields of law. It begins by conceptualizing data as building blocks: It is of little value on its own. But when placed in skillful and creative hands, it can unlock choices for its holders—choices they would not otherwise have—that can generate tremendous profits. Thus, data constitutes what is known as a ‘real option.’ This article shows how using real options to value data can significantly improve upon existing data valuation practices.”

SCOTUS Nominations, By the Nimbers and Over Time

Charles Cameron (Princeton--politics ) & Jonathan Kastellec (Princeton--politics) recently published a book, Making the Supreme Court (OUP), that tracks 90 years of POTUS appointments to the Supreme Court. The Book's description (below) makes clear what the authors attempt to achieve with their book. The authors' execution benefits from data, including on:

• Party and presidential interest in the court, including the use of “litmus tests” for Supreme Court nominees;
• The type of selection processes employed by the president;
• The background characteristics of Supreme Court nominees, and those who made the president’s “short list”;
• Interest group participation in Supreme Court confirmation battles;
• Media coverage of Supreme Court nominees;
• Public opinion of nominees;
• Senate roll call voting on Supreme Court nominees; and
• Simulations on the future ideological trajectory of the Supreme Court.

But what particularly caught my eye was that the authors make available the data (and related coding) that they used in their analyses (click here).

“Appointments to the United States Supreme Court are now central events in American political life. Every vacancy unleashes a bitter struggle between Republicans and Democrats over nominees; and once the seat is filled, new justices typically vote in predictable ways. Making the Supreme Court examines 90 years of American political history to show how the growth of federal judicial power from the 1930s onward inspired presidents, the political parties, and interest groups to shape judicial policy through appointments. The result is a new politics aimed squarely at selecting and placing judicial ideologues on the Court, which has transformed the Court into an ideologically driven and polarized branch. Based on rich data and qualitative evidence, Making the Supreme Court provides a sharp lens on the social and political transformations that created a new American politics.”

Levering AI in the Service of Contract Interpretation

Contract interpretation canons and theories continue to evolve. The latest foray seeks to lever AI.

In a recent article, Generative Interpretation, the authors, David Hoffman (Penn) and Yonathan Arbel (Alabama), advance a new approach designed to assist in determining contracting parties’ meaning. The authors' impulse behind generative interpretation, is, according to them, "simple:" they apply large language models (LLMs) to contractual texts and extrinsic evidence to predict what the parties would have said at contracting about what they meant. The paper's abstract follows.

“We introduce generative interpretation, a new approach to estimating contractual meaning using large language models. As AI triumphalism is the order of the day, we proceed by way of grounded case studies, each illustrating the capabilities of these novel tools in distinct ways. Taking well-known contracts opinions, and sourcing the actual agreements that they adjudicated, we show that AI models can help factfinders ascertain ordinary meaning in context, quantify ambiguity, and fill gaps in parties’ agreements. We also illustrate how models can calculate the probative value of individual pieces of extrinsic evidence.

After offering best practices for the use of these models given their limitations, we consider their implications for judicial practice and contract theory. Using LLMs permits courts to estimate what the parties intended cheaply and accurately, and as such generative interpretation unsettles the current interpretative stalemate. Their use responds to efficiency-minded textualists and justice-oriented contextualists, who argue about whether parties will prefer cost and certainty or accuracy and fairness. Parties—and courts—would prefer a middle path, in which adjudicators strive to predict what the contract really meant, admitting just enough context to approximate reality while avoiding unguided and biased assimilation of evidence. As generative interpretation offers this possibility, we argue it can become the new workhorse of contractual interpretation.”

Lower Fedreal Courts and Religious Claims: An Update

The Supreme Court's ongoing readjustment of long-standing Free Exercise and Establishment clause tensions has renewed scholarly attention to courts' resolutions of religious claims. And in a recent paper, Trump's Lower-Court Judges and Religion: An Initial Appraisal, Stephen Choi (NYU) et al. contribute to a growing empirical literature by levering data on Trump-appointed Circuit Court judges in Free Exercise Clause cases and comparing Trump-appointed judge outcomes with those of Non-Trump Republican- and Democratic-appointed judges.

What the authors find includes that "Trump’s judicial appointments are more religious but no less qualified than other judges," and that "Trump judges are more likely to vote in favor of the plaintiff than not only Democratic judges, but also judges appointed by other Republican presidents."

Notably, however, while Trump-appointed judges vote “pro-religion” (that is, in favor of a religious plaintiff asserting a free exercise claim) more frequently than other judges, this tilt distributes unevenly across various religions. For example, and as the table below illustrates, the authors find “... different voting patterns for Islam. For cases involving Islam, Trump judges vote in favor of Muslim plaintiffs in only 19.2% of the cases compared with 34.5% for Non-Trump Republicans and 48.0% for Democrats.” The authors note that the overall pattern that emerges is “consistent with conventional wisdom: Democrats tend to protect minority religions, and Republicans tend to protect Christianity (and possibly Judaism).”

 

 

The paper's abstract follows.

“It is widely believed that President Donald Trump’s judicial appointments reflected a strategy of appeasing evangelical Christians and other religious groups that favored a more conservative, Christian judiciary, and that in pursuing this strategy Trump sacrificed quality. We explore evidence for this theory by examining the biographies and credentials of Trump’s lower court nominees and the voting records of his judicial appointments to the circuit courts in free exercise cases. We find that Trump’s appointments to the lower courts have stronger or more numerous religious affiliations, and are affiliated with the Federalist Society and the National Rifle Association at unusually high rates, but are no less well credentialed than other judges are. We also find that Trump’s appointments to the circuit court more frequently vote in favor of Christian plaintiffs, and less frequently vote in favor of Muslim plaintiffs, in free exercise cases than judges appointed by other Republican presidents and by Democratic presidents.”

Data on Local Prosecutor Campaign Contributions & Electoral Outcomes

Building on prior descriptive work (here) that focuses on local prosecutor elections in the U.S., in a recent paper, Donating to the District Attorney, Carissa Hessick (UNC) et al., broaden their focus to assess how district attorney campaign contributions map onto electoral success. To inform the authors' interest in "the local politics of mass incarceration," the paper exploits a unique data set that links contribution and election results data. Merging these two data sets allows one to know whether a given recipient [local prosecutor candidate] was an "incumbent or challenger, the recipient’s vote share in both the primary and general election, and whether the election was contested in the primary election, the general election, or either election."

Such data provide a "more nuanced picture of prosecutorial politics, allowing us to make important distinctions between offices and interest groups, suggest new accountability measures, and better evaluate how elections can lead to reform." What the authors find propels them to conclude that "accountability mechanisms beyond elections are necessary to ensure prosecutors do not exercise their significant discretionary power unchecked." The paper's excerpted abstract follows.

“… The study offers a more complete empirical account of prosecutor accountability by analyzing contributions to local candidates as well as their election results. It details the amount of money in local prosecutor elections, including from interest groups, and the relationship between candidate fundraising and success. The stark differences across the country underscore that the more than two thousand local prosecutors are not a monolith; some offices are best understood as political, with contested elections and significant amounts of campaigning, while most appear more bureaucratic...”

Judicial Decisionmaking: Moving Beyond Case Outcomes

While conceding that "judges matter" as a general matter, a recent paper, Do Judges Matter?, considers a slightly narrower question: what about judges matters when it comes to the more mundane aspects incident to the business of judging?" More specifically, the authors, Yun-chien Chang (Cornell) and Geoffrey Miller (NYU), explore potential relationships "between a judge’s personal characteristics and the day-to-day business of courts – the writing of opinions, the use of citations, the frequency of dissent, and like factors."

Exploiting a hand-collected data set of 4,591 substantive decisions issued by 48 state supreme courts in 2003 (excluding Alaska and Hawaii), the paper finds that while state supreme courts themselves vary in how they carry out their judicial business (e.g., citation distributions, frequency of dissents), personal characteristics of judges, by contrast, have only a "negligible impact." The abstract follows.

"An extensive literature examines whether characteristics of judges correlate with votes on cases. These studies generally consider the judges’ votes on the merits of cases. Examining a data set of 4,591 decisions issued by 48 state supreme courts in 2003, we consider whether judges’ personal features affect their opinion writing. We find virtually no significant differences along any of the dimensions under review. Judicial characteristics matter only indirectly — ideological differences increase dissensus in public-law cases, which increases opinion length and citation numbers. Our study suggests that when carrying out the ordinary business of courts, judges are almost all the same."

Data on the Fourth Amendment's "Good Faaith" Exception's Application

While the Fourth Amendment is designed to protect criminal defendants against prosecutors' use of unlawfully obtained evidence, an important exception exists for evidence obtained by law enforcement officers who rely in good faith on existing legal authority. Thus, if police rely on a statute that turns out to be unconstitutional or invalid warrants or precedent, the evidence obtained is nonetheless admissible under the good faith exception.

Empirical analyses on how the good faith exception works "on the ground," however, are scant. To this end, a recent paper, The Reality of the Good Faith Exception, by Matthew Tokson (Utah) and Michael Gentithes (Akron), helps to fill a void.

Their paper draws from available published (and unpublished) judicial decisions (N=5,531) that mention the Fourth Amendment or "motion to suppress" across three separate months (July 2015, 2018, and 2021). Among the paper's findings includes a discernible uptick in the exceptions salience over time (see Figure 1, below).

Figure 1: Good Faith Exception (GFE) Prevalence Over Time - Proportion of Fourth Amendment Suppression Cases Involving the Good Faith Exception

The authors are correctly mindful that selection effects certainly influence their descriptive findings. The authors argue that selection effects in this context likely lead, on balance, to an under-counting of suppression hearings to the extent that "defendants and their attorneys likely decline to file suppression motions because of the good faith exception." (emphasis added)

Overall, the paper argues that "the [good faith] exception, originally confined to narrow circumstances, is turning into a protection for nearly any investigatory activity with an arguable connection to an existing precedent or statute" and that "[c]ourts rely on the exception with remarkable frequency, and a large proportion of courts invoking the exception avoid any substantive constitutional ruling, stunting the development of Fourth Amendment doctrine."

Lay Understandings of Civil Settlements

When it comes to resolving formal civil disputes, popular imagination, partly fueled by the media and entertainment industry, is dominated by "courtroom drama" and jury trials. Empirical work over the decades, however, paints quite a different picture, and one dominated by the "disappearing trial." Thus, as Jennifer Robbennolt (Illinois) et al. make clear in Settlement Schemas: How Laypeople Understand Civil Settlement, "for all the attention paid to the real and imagined drama of the courtroom, these depictions overlook the reality of how many disputes are resolved: through any means but trial."

In the service of seeking a better understanding of how the public understands civil settlements, the authors surveyed a representative sample of more than 1,000 U.S. adults. Key findings include that while the public conveyed a widespread general understanding about civil settlements, evidence of misconceptions about the details, including the role of a jury and settlement scope, also emerged. The paper's abstract follows.

"What does the public think it means to ‘settle’ a civil case? Most legal disputes in the United States end in an agreement to settle, but little is known about what laypeople think about settlement. To fill this gap, we took a direct approach: we asked a nationally representative sample of U.S. adults – more than one thousand of them – basic questions about settlement. We found widespread understanding about the essential nature and frequency of settlement, but persistent, though not universal, misconceptions about the details, including the role of a jury and settlement scope. Because settlement is such a pervasive part of the U.S. legal system, the system’s legitimacy turns in part on how the public understands and views civil settlement. The survey reported here provides a foundational study of the understandings and framework – the schemas – that the public bring to settlement."

Sentencing as a Product of Plea Agreements

When it comes to explaining criminal sentences, conventional wisdom emphasizes either judges or, given that criminal convictions largely flow from negotiated plea agreements, prosecutors. A closer look, however, reveals far more nuance and complexity and a dynamic process involving multiple parties concurrently rather than a static event dominated by one actor.

In a recent "closer look," Jeffrey Bellin (William & Mary) & Jenia Turner (SMU) explore 77 plea bargains in the first half of 2019. Noting that comprehensive empirical work on judicial responses to negotiated plea deals is sparse, In Sentencing in an Era of Plea Bargains makes a few helpful contributions. First, the authors' deep dive into 77 pleas bargains, though descriptive, revels some initial, general contours of these agreements. Notably, while judges ultimately approved all of the 77 plea bargains in the paper's sample, judges departed from the negotiated sentencing recommendation in about one-third of the cases. Also notable is that in virtually all instances where the judge departed from the negotiated sentence, she did so in a direction that favored greater leniency. Second, in separate analyses the authors examine 31 plea agreements that elicited judicial rejections. Somewhat perplexingly, judges rejected most plea agreements (74%) because they felt the recommended sentence was too lenient.

Obviously, descriptive studies that lack statistical power are limited in important ways. At the same time, however, such studies can help by pointing future researchers in potential helpful directions. The paper's (excerpted) abstract follows.

“The literature offers inconsistent answers to a question that is foundational to criminal law: Who imposes sentences? Traditional narratives place sentencing responsibility in the hands of the judge. Yet, in a country where 95 percent of criminal convictions come from guilty pleas (not trials), modern American scholars center prosecutors – who control plea terms – as the decider of punishment. This Article highlights and seeks to resolve the tension between these conflicting narratives by charting the pathways by which sentences are determined in a system dominated by plea bargains.

After reviewing the empirical literature on sentence variation, state and federal plea-bargaining rules and doctrines, and conducting some empirical analysis of our own, we conclude that neither of the competing narratives is correct. Sentencing in the United States has become a dynamic process with substantial contributions from multiple actors, not a static event controlled by any single actor.”

How Summary Judgment Changes Vary Across Subject Areas

The Supreme Court upended summary judgment doctrine with a trilogy of cases in 1986. At the time--and largely persisting into current prevailing wisdom--it was assumed that the 1986 changes made summary judgment comparatively more available and, thus, of greater benefit to defendants. While empirical evidence remains somewhat scant, emerging findings paint a far more complex and nuanced picture.

A recent contribution to this research literature comes from Jonathan Nash (Emory) and Daniel Sokol (USC). In The Summary Judgment Revolution that Wasn't, the authors lever six separate (though related) datasets that draw from federal court decisions in three discrete areas (antitrust, civil rights, and securities regulation) and exploit the 1986 trilogy with pre- and post-intervention comparisons. At bottom, the paper finds "no evidence of the trilogy influencing summary judgment decision making in securities regulation, and at best mixed evidence of such an effect in civil rights cases. In contrast, we find that the trilogy did have a strong statistically significant effect on decision making in antitrust cases, at both the district court and court of appeals levels." The abstract follows.

"The U.S. Supreme Court decided a trilogy of cases on summary judgment in 1986. Questions remain as to how much effect these cases have had on judicial decision making in terms of wins and losses for plaintiffs. Shifts in wins and losses and what cases get to decisions on the merits impact access to justice. We assemble novel datasets to examine this question empirically in three areas of law that are more likely to respond to shifts in the standard for summary judgment: antitrust, securities regulation, and civil rights. We find that the Supreme Court’s decisions had a statistically significant effect in antitrust, an ambiguous effect in civil rights cases, and no effect in securities regulation. We also find that, in the trilogy’s wake, antitrust appellate cases were far more likely to cite trilogy cases—and in particular the one trilogy case that was an antitrust case—than appellate cases in the other areas. This suggests that the lone trilogy case that arose in antitrust had an effect on decision making in that field, but that the trilogy had a limited effect across other substantive areas. This finding differs from Twombly and Iqbal where an antitrust decision ultimately reshaped the entire body of law around motions to dismiss."

Experimental Judicial Decisionmaking Studies & External Validity

For experimental judicial decisionmaking studies, the "gold-standard" research design includes using actual judges as participants. My Cornell colleague (and good friend) Jeff Rachlinski is a leading scholar in this space. For an array of reasons (including practical reasons), too much of the experimental research in this field is hampered by researchers' use of students (including law students) as the experimental participants. To the extant that researchers seek to generalize results from judicial decisionmaking studies to the "universe of judges," conventional  wisdom suggests that the use of students in these experiments raises important external validity questions.

Of course, such (and most) "conventional wisdoms" rest on empirical assumptions. And in a recent paper, Can Law Students Replace Judges in Experiments of Judicial-Making?, Holger Spamann (Harvard) and Lars Klohn (Humboldt) submit this question to an experiment. The answer to the question they pose in their paper's title?: "unfortunately, no." The paper's abstract follows.

"Experimental research on judicial decision-making is hampered by the difficulty of recruiting judges as experimental participants. Can students be used in judges’ stead? Unfortunately, no. We ran the same high-context 2×2 factorial experiment of judicial decision-making focused on legal reasoning with 31 U.S. federal judges and 91 elite U.S. law students. We obtained diametrically opposed results. Judges’ decisions were strongly associated with one factor (sympathy, i.e., bias) but not the other (law). For students, it was the other way around. Equality between the two groups is strongly rejected. Equality of document-view patterns—a proxy for thought processes—and written reasons is also strongly rejected."

Specific Performance, Expectations, and Elon Musk

For one brief moment in time in 2022, to some it looked as though a Delaware judge was poised to order Elon Musk to purchase Twitter, over his objections, due to a contract breach. Had that happened, as Tess Wilkinson-Ryan (Penn) et al. note in Expecting Specific Performance, it would have been "the highest-profile specific performance order in living memory."

The incident helped prompt Wilkinson-Ryan and colleagues to explore intuitions on specific performance through a series of mixed-method experiments. And what they find, on balance, are "fragile" intuitions. Levering a mix of survey and experimental data, the authors find that "ordinary people think that courts will give them exactly what they bargained for after breach of contract; in other words, specific performance is the expected contractual remedy." They go on to argue that "the default expectation of equitable relief is a widespread but malleable intuition—and that even a fragile legal intuition has practical consequences, individually and systemically." Indeed, updating subjects with the prospect of money damages for a contract breach loosens initial default expectations for specific performance. The paper's abstract follows.

“Using a series of surveys and experiments, we find that ordinary people think that courts will give them exactly what they bargained for after breach of contract; in other words, specific performance is the expected contractual remedy. This expectation is widespread even for the diverse array of deals where the legal remedy is traditionally limited to money damages. But for a significant fraction of people, the focus on equity seems to be a naïve belief that is open to updating. In the studies reported here, individuals were less likely to anticipate specific performance when they were briefly introduced to the possibility that courts sometimes award damages in contract disputes.

We argue that the default expectation of equitable relief is a widespread but malleable intuition—and that even a fragile legal intuition has practical consequences, individually and systemically. In a follow-up experiment, we show that subjects are more interested in the prospect of efficient breach when they know that money damages are a possible remedy. This finding suggests that the mismatch between what people assume the law will do (specific performance) and what it actually does (money damages) sometimes encourages performance. We consider the potential for exploitation of this tendency. Finally, we offer some suggestions about how law and psychologists should elicit folk beliefs about legal rules and remedies."

Criminal Records & Accuracy

Given the outsized implications for individuals, to note that criminal record keeping requires accuracy is to note merely the obvious. This is especially true in the employment context where "94% of employers conduct background checks" on job applicants. Growing employer demand for background checks has, in turn, fueled a dramatic increase in private sector service providers in this space. While access to and uses of criminal records have expanded, concerns persist about "serious data quality problems" in various criminal justice databases.

To explore this issue, in a recent paper, The Problem With Criminal Records, Sarah Lageson (Rutgers--crim. just.) & Robert Stewart (Maryland--crim. just.) compare criminal record reports for 101 New Jersey participants, all of whom had at least one criminal change in their history, across three sources: (1) the official state record; (2) a consumer reporting agency; and (3) an open source, public records data aggregator. The researchers then asked each of the 101 participants to review and audit the information retrieved from the three sources.

As the figure above visually implies, what the researchers found includes errors, and lots of them. Among the 2,406 individual charges reported by the three separate sources, only 8.3% of the participants' criminal charges (200/2,406) were accurately reported by all three sources. Moreover, 60% and 50% of participants had at least one false positive error on their regulated and unregulated background checks, and nearly all (90% and 92% of participants, respectively) had at least one false negative error. As this study is limited to individuals with criminal histories, it remains unclear whether errors also implicate those with no criminal history. The paper's abstract follows.

"Criminal records are routinely used by employers and other institutional decision-makers, who rely on their presumed fidelity, to evaluate applicants. We analyze criminal records for a sample of 101 people, comparing official state reports, two sources of private sector background checks (one regulated and one unregulated), and qualitative interviews. Based on our analysis, private sector background checks are laden with false positive and false negative errors: 60% and 50% of participants had at least one false positive error on their regulated and unregulated background checks, and nearly all (90% and 92% of participants, respectively) had at least one false negative error. We define specific problems with private sector criminal records: mismatched data that create false negatives, missing case dispositions that create incomplete and misleading criminal records, and incorrect data that create false positives. The prevalence of error in private sector background checks limits access to social opportunities ranging from employment, education, and housing."

Modeling Rare Events--SCOTUS Clerkships

Of the approximate 36,000 law school students who graduate each year, only 36 or so end up securing a SCOTUS clerkship. While a handful of biographical and anecdotal accounts exist, very few systematic data exist on the "1-in-1,000" clerkship lottery.

A recent paper by Tracey George (Vanderbilt), Albert Yoon (Toronto), and Mitu Gulati (Virginia), Some Are More Equal Than Others: U.S. Supreme Court Clerkship, digs into 40 years worth of data and provides rich granularity about this statistical rare event. In the main, while some unexpected twists emerge, what the authors find generally comports with perceived wisdoms. For example, a disproportionate number of clerks emerge from five elite law schools. One interesting change, however, is that while a SCOTUS clerkship once led "primarily to government and universities [law faculties]," former clerks now increasingly land at "law firms and corporations." An excerpted abstract follows.

"... Yet, we lack a clear story about who wins this clerkship lottery. This Essay seeks to provide that story. Our analysis relies on new datasets of all Court clerks who served between 1980 and 2020, including the details of their path to the Court and their road after. We amend and expand on theories of success in this important labor market. All clerks are highly qualified, yet the same is true for many not chosen. We find that educational pedigree, as opposed to academic performance or any other qualification, often distinguishes the winners from the also-rans. The Court clerkship selection process, like other elite labor markets, proves to be a blend of status and merit where status often prevails. Our analysis does not end there, however. We go on to look at where this forty-year cohort is currently working and confirm that once attained, a Court clerkship does lead to a bounty of opportunities including a return to the Court as a justice. Thus, the Court clerkship lottery is an important labor market not only to lawyers but also to society writ large. In the elite legal labor market, some people are, in fact, more equal than others."

More Evidence of a Racialized Death Penalty

Notably, race does not emerge as a central concern in the Court’s Furman decision, now more than fifty years old. In the decades since Furman, however, empirical evidence continues to mount documenting that "racial discrimination infects the administration of the death penalty." A recent paper by Jeffrey Fagan et al. (Columbia), Getting to Death: Race and the Paths of Capital Cases after Furman, represents a recent and important contribution to this literature. Researchers visited each judicial circuit in Georgia to record the details of every murder case that entered the courts from 1994–2005. Analyses of the 1,317 death-eligible murders suggest that “race continues to matter, and that the statutory aggravators that distinguish death cases from other killings are themselves racialized.” A more detailed description of the paper's core findings appears in the Abstract (below).

“Decades of research on the administration of the death penalty have recognized the persistent arbitrariness in its implementation and the racial inequality in the selection of defendants and cases for capital punishment. This Article provides new insights into the combined effects of these two constitutional challenges. We show how these features of post-Furman capital punishment operate at each stage of adjudication, from charging death-eligible cases to plea negotiations to the selection of eligible cases for execution and ultimately to the execution itself, and how their effects combine to sustain the constitutional violations first identified 50 years ago in Furman. Analyzing a dataset of 2,328 first- degree murder convictions in Georgia from 1995–2004 that produced 1,317 death eligible cases, we show that two features of these cases combine to produce a small group of persons facing execution: victim race and gender, and a set of case-specific features that are often correlated with race. We also show that these features explain which cases progress from the initial stages of charging to a death sentence, and which are removed from death eligibility at each stage through plea negotiations. Consistent with decades of death penalty research, we also show the special focus of prosecution on cases where Black defendants murder white victims. The evidence in the Georgia records suggests a regime marred less by overbreadth in its statute than capriciousness and randomness in the decision to seek death and to seek it in a racially disparate manner. These two dimensions of capital case adjudication combine to sustain the twin failures that produce the fatal lottery that is the death penalty.”

Data on Whether Lawyers Add Value in IPOs

A common complaint is that lawyers provide the "grit" while the I-bankers provide the "grease" when it comes to commercial transactions, especially IPOs. Existing research on lawyers' transactional roles focuses on two alternative perspectives: The skilled advice and the passive execution hypotheses. To test these two competing hypotheses, in a recent paper, Do Lawyers Matter in Initial Public Offerings?, Bates (ASU--Finance) et al. draw from the Thomson Financial Securities Data Company (SDC) new issues database to identify U.S. IPOs over the period of 1986–2016. After filtering, the study's final sample includes data on 7,460 IPOs.

What the paper finds may--or may not--surprise. Specifically, the results imply that issuer law firm fixed effects account for an "economically significant portion of the variation in IPO outcomes, even after accounting for underwriter fixed effects." Specifically, the paper notes that "legal advisor fixed effects explain approximately 16.7% of the variation in IPO underpricing, which is about 70% of the variation attributable to underwriters documented in this and prior studies." Thus, if these effects are real, the paper evidences "the economic importance of legal advisors in the IPO process." The abstract follows.

"This study provides novel evidence that law firms representing issuers have a material impact on economic outcomes in the IPO market. Fixed effects for legal counsel explain a significant portion of the variation in IPO underpricing, comparable to underwriter fixed effects documented in prior studies. Issuer law firms also exhibit differential skill in moderating the likelihood of IPO-related litigation, and have an idiosyncratic influence on the structure of IPO disclosures. Accounting for endogenous selection, we document that law firms associated with lower rates of IPO-related litigation are also associated with less first-day underpricing and receive higher fees from their clients."