Empirical Legal Studies

From PACER to SCALES: Access to Federal Court Data

Even basic research questions on Art.III courts and federal litigation typically confront consequential barriers to needed data (e.g., federal court records). While the federal government's PACER platform and a few third-party operations provide some relief, both possess their own respective shortcomings and limitations (including, but not limited to, cost).

Initial fruits of a new collaborative effort based at Northwestern University, SCALES, have emerged recently. SCALES is an online platform that seeks to "assemble federal court records, systematically organize them and extract key information, and—most importantly—make them freely available to the public." What exists now includes access to information and records for all federal cases initiated in 2016 and 2017. As SCALES' mission includes "to make court records freely accessible to the public," the platform aspires to, in the future, expand coverage to include "all years."

In a recent paper, The SCALES Project: Making Federal Court Records Free, the authors, David Schwartz (Northwestern) et al., explain the SCALES initiative in greater detail. As well, the project also benefits from its own website (click here).

Initial Data on Trump Judges' Performance

Persistent general public complaints about Trump's Art.III judicial picks include that they are "unqualified" and were drawn from an unduly narrow pool of traditionally plausible candidates. Of course, and as they say, "where data are scant, 'opinions' bloom." In a recent recent paper, How Different Are the Trump Judges, Stephen Choi (NYU) & Mitu Gulati (UVa) bring some initial data to these prevailing conventional wisdoms. The paper sets out to examine how Trump-appointed judges "have done compared to other presidents' appointments." Their findings challenge prevailing and widely-held assumptions.

The paper includes all active circuit court judges between January 1, 2020, through June 30, 2023. While the analyses are largely descriptive, the paper explores judge performance in terms of three key metrics: productivity, influence, and the degree to which a judge is willing to "deviate" from other judges, and particularly from "those closest to them in terms of political ideology." As the paper's abstract (below) implies, and contrary to public predictions of underperformance, the authors find that "Trump judges outperform other judges."

"Donald J. Trump’s presidency broke the mold in many ways, including how to think about judicial appointments. Unlike other recent presidents, Trump was open about how 'his' judges could be depended on to rule in particular ways on key issues important to voters he was courting (e.g., on issues such as guns, religion, and abortion). Other factors such as age and personal loyalty to Trump seemed important criteria. With selection criteria such as these, one might expect that Trump would select from a smaller pool of candidates than other presidents. Given the smaller pool and deviation from traditional norms of picking 'good' judges, we were curious about how the Trump judges performed on a basic set of measures of judging. One prediction is that Trumpian constraints on judicial selection produced a different set of judges that underperform compared to judges appointed by other presidents. Using data on active federal appeals court judges from January 1, 2020 to June 30, 2023, we examine data on judges across three different measures: opinion production, influence (measured by citations), and independence or what we refer to as “maverick” behavior. Contrary to the prediction of underperformance, Trump judges outperform other judges, with the very top rankings of judges predominantly filled by Trump judges."

Judicial Decisionmaking "In the Time of COVID-19"

As the scope and reach of the COVID-19 pandemic's impact remain difficult to over-state, it will surprise few to learn that the full reach of public health officials' legal/statutory claims did not escape judicial scrutiny. What is not yet well understood, however, is a more granular understanding of how legal decisions informed particular legal doctrines, including those involving religious liberty and statutory interpretation in the public health silo. To this end, a recent paper by Michelle Mello et al., Judicial Decisions Constraining Public Health Powers During COVID-19: Implications for Public Health Policy Making, takes an initial empirical cut.

Deploying "legal epidemiology," the paper analyzes 112 federal and state courts decisions between March 2020 and March 2023 that involve successful plaintiff challenges to federal, state, or local pandemic-motivated "mitigation" orders. The authors argue that the 112 cases reveal "areas of profound instability in how courts analyze challenges to exercises of public health legal powers." The disruptions were particularly acute in the religious liberty and statutory authority areas. The findings make clear that going forward assertions of public health authority will unlikely benefit from "broad [judicial] deference." The paper's abstract follows.

"Public health legal powers are increasingly under pressure from the courts in the United States. During the COVID-19 pandemic, individuals and organizations successfully challenged many community mitigation orders (for example, mask mandates, vaccination mandates, and restrictions on gatherings), demonstrating the legal vulnerability of disease control measures. Analyzing 112 judicial decisions in which the plaintiff prevailed from March 2020 through March 2023, we examined the ways in which courts constrained public health powers during the COVID-19 pandemic. We found that in these 112 decisions, courts shifted how they analyze religious liberty claims and reviewed challenges to the exercise of statutory powers by health officials in novel ways. We discuss implications for public health policy going forward, and we recommend ways in which legislatures and health officials can design policies to maximize their prospects of surviving legal challenges."

Municipal Revenue Generation and Enforcement Incentives

While an emerging literature explores possible links between government revenue generation and enforcement incentives, much of the literature is theoretical. Even though conflicts of interest are rife in this context, empirical testing is comparatively scarce.

In a recent paper, Agency Incentives and Disparate Revenue Collection: Evidence from Chicago Parking Tickets, Elizabeth Luh (SEC) et al. explore whether a recent increase in motor vehicle registration fees as well as related fines for non-compliance in Chicago informed the Chicago Police Department's ("CPD") parking enforcement patterns. By increasing both the cost of motor vehicle registration and the fine for non-compliance in 2012, the City of Chicago simultaneously made vehicle registration compliance more expensive and enforcement more profitable relative to other parking fines.

The paper executes a difference-in-differences (DiD) strategy to assess the relative change in sticker enforcement across various types of neighborhoods, "focusing on Black versus non-Black areas due to the purported claims of disparate impacts from the general public." The paper reports that CPD vehicle registration enforcement increased by nearly 2,500 tickets in Black relative to non-Black neighborhoods per year. It also finds that "non-sticker ticket enforcement significantly increased by nearly 12,000 tickets, despite the non-sticker fines remaining largely the same." The authors interpret these patterns as evidence consistent with a broader revenue collection effort. The paper's abstract follows.

"We leverage a sharp 2012 parking fine increase for failing to purchase vehicle registration to examine disparate ticketing patterns across enforcement agencies in Chicago. Using an event-study framework, we find that Chicago police increased their enforcement of car registration non-compliance in Black relative to non-Black neighborhoods, with no observed disparate response for non-police enforcement agencies. This disparity is unexplained by differences in non-compliance and is instead driven by departmental revenue incentives and lower marginal search costs in Black neighborhoods. Disparate enforcement also exacerbated existing gaps in financial instability, including increased rates of ticket non-payment and bankruptcy filings in Black neighborhoods."

Farmland Property Transfers Across Generations in the Mid-20th Century

Farmland property succession is a central issue in agricultural policy, and succession in the agricultural sector is quickly becoming even more salient owing to the number of farmland transfers expected in the coming years. Nevertheless, and owing to a paucity of data, the scarcity of research on farmland property transfers across generations is, according to Adrian Haws (Cornell-Econ.) et al., “jarring.”

In their recent paper, Who (Actually) Gets the Farm? Intergenerational Farm Succession in the United States, the authors link Census microdata for census records for 1.3 million farmers in 1900 or 1910 to their children’s succession outcomes up to 40 years later. Such a data set provides glimpses into which farmers’ children, if any, took title to farmland property.

A within-family identification strategy yielded some notable findings, including that daughters were rarely successors and evidence of a small and monotonic birth order effect among sons. An excerpted abstract follows.

“Farm succession is a central issue in agricultural policy. Yet while many studies explore succession planning, little is known about how farms are actually transferred. … Using a within-family identification strategy, we find that first-born sons are slightly more likely than their younger brothers to be successors while their parents are working aged. However, birth order is not predictive of who receives the farm when parents are older or deceased. For later farm transfers, sons who were previously tenant farmers are much more likely than their brothers to be successors, possibly because they are better prepared. Fewer than one-fifth of farmers transfer their farm to any son….”

Birth Order Effects in Developing Countries

While birth order effect studies' popularity has waned a bit over time, most focus on developed countries. The consistently negative results in developed countries imply that "… the later a child is born within a family, the worse their adult economic outcomes relative to their earlier- born siblings” incident to potential intra-familial (small- or capital D) discrimination. By contrast, the comparatively smaller number of studies that use data from developing countries yield unstable results.

A recent paper, Birth Order Effects on Education: Insights from Low- and Middle-Income Countries, sets out to fill a scholarly void involving involving emerging nations. In it, Silvia Mendolia (Univ. Turin) et al., estimate mothers’ fixed effects models assessing the impact of birth order on schooling and school attendance. The models exploit "almost 1.8 million observations gathered from the Demographic and Health Survey (DHS) to measure birth order effects on children’s educational outcomes in 35 developing countries, between the mid-1980s and 2020."

Overall, the paper finds evidence of negative birth order effects on educational attainment in 32 out of 35 developing countries. Notably, the paper finds that in most developing countries, “overall birth order effects do not vary by gender, family wealth, location or over time.” The paper's (excerpted) abstract follows.

"... To the best of our knowledge, this is the first study analyzing birth order effects in a comprehensive set of developing countries. In developing countries, families tend to be relatively large and within-family resources scarce. The DHS contains harmonized data and variables for all countries, providing a picture of birth order effects that is consistent across the developing world. Using mothers’ fixed effects models, we estimate the impact of birth order on standardized years of schooling and school attendance, exploring non-linearities in birth order effects, as well as heterogeneous effects by gender, socio-economic characteristics and over time. Overall, we find negative birth order effects on educational attainment in 32 out of 35 countries. Consistent with this, we find that the probability of not being in school increases with birth order. We find that in most countries, overall birth order effects do not vary by gender, family wealth, location or over time...."

Data on Police Separations

While national news and policymakers focused on a surge in police separations in many large city departments and identified social unrest incident to the George Floyd murder as a likely contributor, much of the public discussion unfolded amid a paucity of data. By contrast, a recent paper by Ben Grunwald (Duke), A Large-Scale Study of the Police Retention Crisis, sets out to submit prevailing conventional wisdoms to data.

To do so, the paper exploits data drawn from 6,800 local law enforcement agencies across 15 states that, taken together, serve one-half of the U.S. population. As the Figure (below) descriptively implies, the acceleration in police officer separations beginning in the summer of 2020 was not off-set by any increasing in police hiring.

While what to infer from these two trend lines is not obvious, the paper lands on two main descriptive findings. First, "the increase in separations in IPED agencies after the summer of 2020 was smaller, later, less sudden, and possibly less pervasive than the retention-crisis narrative suggests. All told, the cumulative impact on the labor force by the end of 2021 was just 1%." Second, these "[a]ggregate figures, however, mask variation at the agency level. As I show, a substantial minority of large agencies meaningfully shrank by the end of 2021.”

JLEA

Further evidence of a healthy, robust, and growing scholarly field is provided by the emergence of an important new online journal in the ELS space. Co-edited by Christoph Engel (Max Plank Institute), Lee Epstein (USC), Daniel Klerman (USC), and Eyal Zamir (Hebrew) and published by Sage, the Journal of Law & Empirical Analysis (JLEA), a peer-reviewed, open access journal, "aspires to spearhead the empirical turn in law by publishing cutting-edge research that is accessible and of interest to the wider legal community. JLEA aims to attract papers written by social scientists and legal academics alike that have clear legal relevance, that appeal to an international audience, and that combine theory and empirics, whether to draw inferences or to offer rich description."

Information on its inaugural issue can be found here.

Federal Court Judicial Opinons and Judge Gender

For years (decades, even), legal and political science literatures have considered whether judge gender informs judicial outcomes. While the standard interpretation emphasizes that judicial attributes (including judge gender) play a causal role, an alternative explanation considers the possibility that case characteristics (and therefore potentially outcome) become associated with judicial attributes via some other mechanism, such as case assignment, or decisions about what cases to publish or report.

Results in a recent paper, Gendered Judicial Opinions, by Michael Livermore (UVa) et al., substantially support an assignment/selection pathway explanation. To be sure, the paper is appropriately cautious cautious and notes that while its findings support the assignment/selection pathway, the possibility of a gender causation pathway cannot be properly excluded as the two are not mutually exclusive.

The paper levers a rich data set that includes all federal circuit court opinions, published and unpublished, from 2001 through 2017, except for decisions of the Court of Appeals for the Federal Circuit, en banc opinions, and opinions that include fewer than 900 characters. The authors then merged textual data from judicial opinions with judge-level demographic information collected by the Federal Judicial Center (FJC). The paper's abstract follows.

“In this paper we investigate whether gender is associated with the content of judicial opinions in the U.S. courts of appeals. Using a topic model analysis, we find that gender is a significant predictor of the content of judicial opinions. Two causal pathways could explain this result: (1) men and women judges write differently about the same cases; (2) men and women judges write about different cases, either due to assignment or selection effects. To untangle these two pathways, we carry out three additional analyses. First, we examine whether the United States as a party is associated with judge gender. We next examine whether case codes are associated with judge gender. Finally, we examine the relationship between topic prevalence and gender, controlling for case codes. Our findings lend greater support to the second pathway than the first. This result raises the prospect that prior work on gender-based differences in judicial behavior may be confounded by assignment or selection effects. Our results also raise normative concerns about gender disparities in voice and influence in the U.S. courts.”

Regulation & 3rd-Party Litigation Financing

As the world of 3rd-party litigation financing ("LTPF") grows, state efforts to regulate this industry sector endeavor to keep pace. While much of this activity takes place outside of public view, scholarly attention is increasing.

One example includes a recent paper by Ronen Avraham (Tel Aviv, Texas), Anthony Sebok (Cardozo) & Joanna Shepherd (Emory), The Whac-A-Mole Game: An Empirical Analysis of the Regulation of Litigant Third Party Financing. What the paper finds, unsurprisingly, is that as states ramp up regulation on LTPF, funding activity declines; where loosening of state regulations corresponds with increased funding activity.

Another core finding, and inspiration for the paper's title, is that funders continually adjust contract terms in an effort to circumvent regulations. To illustrate, "when legislation prohibits compounding or limits how long funders can charge interest, the funder responds by increasing the posted monthly interest rate. Although these interest rate increases are not enough to completely offset the impact of the other restrictions on the funder’s returns, the funder nevertheless has higher returns than it would have without the attempted circumvention." Thus, akin to the "classic Whac-A-Mole game, funders try to maintain their pre-regulation per-funding-return no matter how states’ regulatory activity tries to get rid of it."

The Empirics of Rental Evictions in Kentucky

As statutory eviction pauses implemented during the COVID-19 pandemic expire, eviction rates are beginning to rebound to pre-pandemic levels. Though this issue is fraught with an array of legal and policy implications, the topic is also noted for a paucity of empirical research. A recent paper, Rural Renting: An Empirical Portrait of Eviction, by Cassie Chambers Armstrong (Louisville) & Christopher J. Ryan, Jr. (IU-Bloomington), makes a helpful contribution by focusing on rental eviction outcomes in Kentucky. Specifically, the paper analyzes factors "influencing the eviction process in the rental market—and the likelihood that it will result in a judgment against a renter— once a property owner initiates legal proceedings."

To this end, the paper levers data gathered by the Kentucky Administrative Office of the Courts (“KOC”) from all 120 Kentucky counties from 2018-2022, yielding usable data on 202,572 renter evictions. One data set limitation is that it excludes information on pre-filing interactions and negotiations between renters and property owners.

Key findings include that those living in rural areas were more than 55 percent more likely to experience a judgment of eviction just by virtue of where they lived and those living in jurisdictions that had adopted renter-friendly policies (URLTA) were more likely to avoid a judgment of eviction—even though these laws do not impact the eviction process itself. Interestingly, renters who lived in the same ZIP code and/or state as their landlord were significantly more likely to resolve their case prior to judgment. An excerpted abstract follows.

"... Collectively, these findings, and others, have implications for eviction policy. They suggest that we must pay more attention to rural courts and the factors that lead to disparate outcomes in them. They call for programs that encourage collaboration between property owners and home renters, such as mediation and other types of eviction diversion. These results also must lead to further study of the time period between when an eviction case is filed and when it is concluded to more fully understand how the various parties approach decision-making with this legal process...."

Initial Data on Data Breach Litigation

News reports about massive data breaches are quickly becoming all-too common. While states have adopted data breach notification statutes and an increasing number of privacy statutes, litigation in this space remains largely by common law doctrines. Despite steady increases in litigation over time, quite little is know, systematically, about how such litigation has unfolded.

At the risk of stacking "turtles all the way down," a recent paper, Unto the (Data) Breach, sets out to gather initial data on data breach litigation. Drawing from WestLaw and PACER documents, the paper identifies 225 cases (between 2005 and 2022). While the analyses are exclusively descriptive, this is representative of helpful granular foundational work from which more technically sophisticated analyses can flow in the future. One take-away from the paper is that a comparative absence of a "cohesive data breach common law" likely exerts upward pressure on settlement activity. An excerpted abstract follows.

“Since the early 2000s, U.S. courts have begun hearing “data breach” liability cases, the inevitable result of a growing internet-connected technology infrastructure. The relatively recent development of case law signals a body of law in development, stunted by significant limiting factors that prevent the coalescence of legal principles. … This descriptive empirical study analyzes, in detail, 225 data breach cases from 2005-2022, reviewing these cases over a three-year period to descriptively identify key trends and changes within a case’s life on the docket.

This study identifies the type of plaintiff, settlement amounts, type and disposition of information compromised, claims, common motions, and key strategies most likely to result in a favorable plaintiff outcome. It also explores broad trends in data breach litigation, including acceptance of claims by courts, the status of future harms in 12(b)(1) and 12(b)(6) standing challenges, and the degree to which courts are willing to let cases proceed beyond preliminary motions….”

Data on U.S. Solicitor General's "Flips"

While many criticize the Supreme Court for "unsettling" Court precedent, exceptionally few focus on instances where the Solicitor General's Office ("SG") "advances legal arguments that differ from those advanced by government lawyers in previous cases." Notably, '“[e]ven those observers who defend the SG, including veterans of the office, caution that inconsistency in legal argument poses a threat to the SG’s credibility with the Court." A recent paper by Margaret Lemos (Duke) & Deborah Widiss (Indiana), The Solicitor General, Consistency, and Credibility, sets out to empirically better understand the circumstances that lead the SG to "change its position on the meaning of the law, and to unpack the connections between consistency and credibility.”

To do so, the paper leverages an original dataset that includes 131 cases drawn from 1892 to the Court's 2022 term. The authors make clear that the sample is under-inclusive in that it contains only those SG "flips" that are "evident": that is, where "the government itself, other parties or amici, or the justices flagged that the government had changed its legal position." In about 80% of the cases, the SG itself acknowledged the change in its briefing. Of course, that also means that in 20% of the cases the government "either did not discuss, or denied, the changed position."

While the paper's analyses are strictly descriptive, main findings include that changes in the SG's litigating positions have increased over time. The authors also emphasize that SG reversals (or "flips") happen for an array of (often overlapping) reasons, "many of which stem from the SG’s unique role in coordinating litigation across a vast and constantly changing federal government." An excerpted abstract follows.

“This Article offers the first comprehensive look at cases in which the Solicitor General (SG) rejects a legal argument offered on behalf of the United States in prior litigation. Such reversals have received considerable attention in recent years, as shifts in presidential administrations have produced multiple high-profile “flip-flops”—as the justices sometimes call them—by the SG. Even those observers who defend the SG, including veterans of the office, caution that inconsistency in legal argument poses a threat to the SG’s credibility with the Court. Our goal is to better understand the circumstances that lead the SG to change its position on the meaning of the law, and to unpack the connections between consistency and credibility.

… Indeed, our study calls into question the idea that ideological swings associated with changes of presidential administrations can be isolated, either in theory or in practice, from other sorts of legal, social, and technological changes that shape the government’s understanding of the law. It also shows that the connection between consistency and credibility, while intuitive at first blush, rests on a formalist understanding of law and an unpersuasive equation of the judiciary and the executive.”

COVID-19 Relief Funding: Following the Money Across States

In response to the global COVID-19 pandemic, the U.S. federal government, through the CARES Act, FFCRA, RRA, and ARPA, provided as much as "$6 trillion in income support to households, a combination of loans, grants, and tax relief to firms and non-profits, funding for (public) health efforts, and intragovernmental grants to sub-national governments." These federal funds included "roughly $900 billion in aid to state and local government entities." And a recent paper, What Do State Do With Fiscal Windfalls? Evidence from the Pandemic, by Jeffrey Clemens (UCSD--econ.) et al., focuses on how state governments deployed this federal aid.

To do so, the paper benefits from a unique dataset constructed from a rich array of sources and an estimation strategy that levers variations across states with "low vs. high levels of per resident representation in the U.S. Congress." More specifically, owing to the possibility that federal aid to states may be endogenous along many dimensions, the paper deploys "an instrumental-variable approach to estimate the effect of federal relief to state and local governments on the revenue and expenditure decisions of state governments."

Among the findings include that "for every $1000 of committed federal aid per capita, $645 appears as increased intergovernmental grant revenue in states over the years 2020-2022." When it came to deploying the federal funds, state governments emphasized "General Government Expenditures ($379 per $1000 of federal aid) and state and local pension contributions ($72 per $1000 of federal aid)." An excerpted abstract follows.

“Using variation in federal pandemic-era fiscal aid to states driven by the strength of political representation, we find that incremental pandemic-era fiscal aid to states was most likely to end up in the categories of general administrative service spending and employee pension benefit funding. Spending on categories that motivated the aid in the first place, such as healthcare, education, and infrastructure, may also have responded but does not show robust patterns. Total state government revenues and expenditures had increased by around 70 cents per incremental windfall dollar of committed federal funds by 2022. Of this, the statistically significant categorical spending effects are 38 cents to general government expenditures (the residual that in principle excludes healthcare, education, infrastructure, and other functional categories) and 7 cents to pension funding, even though the latter use was inconsistent with the objectives of the legislation.”

"Too-Big-To-Fail" Banks, Bailouts, and Fairness

While such questions as "are regulators across countries 'in the pockets' of large banks?" and "do supervisors provide favorable treatment when bailing out large, global systemically important banks?" persist, conventional wisdom strongly implies that "regulators around the world give [Too-Big-To-Fail] TBTF banks preferential treatment in bailouts relative to other banks, often viewed as an unfair advantage for these banks."

While an unfair regulatory advantage tilting in favor of TBTF banks has "been engrained in the minds of banking researchers, policy makers, and the public around the world for decades," to date little empirical work has subject this belief to data. In a recent paper, Do “Too-Big-To-Fail” Banks Receive Preferential Treatment in Bailouts? Surprising Results from a Cross-Country Analysis, Allen Berger (Univ. S. Car.—Business) et al. do just that and find evidence that "soundly reject(s) implications of unfair preferential treatment concept."

The authors lever hand-collected bank bailout data in the EU from 2008-2014. Their sample includes 100 banks whose "cumulated total assets represent about 50 percent of the total assets of EU banks at the end of 2014" and "35 bailout events for capital injections and 61 bailout events for debt guarantees. Among these, 10 capital injections and 24 debt guarantee bailouts are provided to large, systemically important banks.”

The paper's findings largely contradict prevailing conventional wisdom. "We find statistically and economically significant evidence that regulators treat [Globally-Systemically Important Banks] G-SIBs more harshly than other banks. Regulators make G-SIBs wait longer for bailouts until their capital ratios have more significantly deteriorated, apply more severe restrictions to them, and withdraw government bailout support sooner at lower capital ratios than for other banks in their countries.” The abstract follows.

"Regulators more often bail out “Too-Big-To-Fail” banks than others, but this may not imply preferential treatment as commonly believed. Bailouts are complex dynamic processes involving more than one-time aid, so harsh treatments elsewhere in the process may counter the benefits of the higher likelihood of bailouts for these banks. Using bailout data from 22 European countries we find relatively harsh treatment for Globally-Systemically Important Banks. Regulators bail out G-SIBs at later stages of financial deterioration, impose stronger restrictions, and withdraw aid after less significant recoveries. We explain these findings using cross-country data on supervisory powers, political connections, and national culture."