Empirical Legal Studies

Selection Effects & Prof. George Priest (1947-2024)

A prior post noting Prof. George Priest's passing predictably emphasizes his enduring influence on the topic of selection effects in litigation. Among the many papers scheduled for publication in 42:3 Yale J. on Reg. (2025) and presented at a symposium (or 'Festschrift') just months before Priest's recent passing includes 'The Selection of Disputes' at Forty, by Daniel Klerman (USC) & Yoon-Ho Alex Lee (Northwestern). The Klerman & Lee paper (abstract below) complements, contributes to, and reflects Priest's enormous scholarly legacy.

“Priest and Klein's ‘Selection of Disputes for Litigation’ has been one of the most influential law review articles of all time. This Article reviews its contribution to legal scholarship. Priest and Klein's central and enduring contribution is the recognition that some cases are more likely to settle than others. It follows that litigated disputes are not a random sample of all disputes. This basic insight is true under nearly all litigation models and is also confirmed by a large body of empirical evidence. Priest and Klein's article is also famous for its prediction that, under certain conditions, the plaintiff trial win rate will tend to approach fifty percent. That prediction, however, is not supported by most other litigation models and has received only modest support from the empirical literature. Our citation analysis also suggests that 'The Selection of Disputes for Litigation' is one of the rare articles whose importance was both recognized almost immediately and whose influence has continued to grow over several decades.”

Empirical Evidence, Courts, and the 4th Amendment

Over time, and since Katz (1954), Justice Harlan's two-part approach has largely guided how courts construe privacy protections in the 4th Amendment context. As the second part of Harlan's approach involves what many consider to be amenable to empirical social science, a growing body of scholarship sets out to mark the boundaries of where, as Christine Scott-Hayward (CSU-Long Beach--criminology) et al., note, "society [reasonably] expects privacy." Increasingly, defense attorneys increasing turn to such research findings in their briefs and pleadings to courts.

While entirely descriptive, The Role of Empirical Scholarship in Fourth Amendment Privacy Jurisprudence levers 13 leading empirical papers in the 4th Amendment space and then identifies and dives deeply into the relatively (and absolutely) small number of judicial opinions that cite to the leading empirical work. The authors' main takeaway is one of frustration with the courts' reluctance to engage with empirical legal scholarship relevant to the question of societal expectations of privacy in the search and seizure context. (To be sure, any judicial reluctance to engage with empirical legal research is merely a sub-set of, for better or worse, courts' collective benign neglect of legal scholarship generally.)

In the end, the paper's findings (and the authors' frustration) both bump into and provide indirect support for a larger point that my Cornell colleague (and friend) Jeff Rachlinski described years ago (2011): “Empirical legal scholarship . . . is not the same as evidence-based law. Empirical legal scholarship resides primarily in the academy.” That relevant empirical legal scholarship will migrate from the academy and into judicial opinions is not a given. Moreover, any such migration will likely continue to vary across legal doctrines and case types.

SCOTUS Database--Updates

The Supreme Court Database, motivated in part by Prof. Harold Spaeth (1930-2017), remains a leading source for Court and judicial decisionmaking scholars. Indeed, "virtually all systematic analyses of the contemporary Supreme Court and its members have relied on it." Given the Database's scholarly prominence, I wanted to pass along a few, brief updates.

First, the Database, which includes information (247 variables) on every SCOTUS decision since the "beginning" (1791), has now been updated to include information on all SCOTUS cases through the 2023 Term. Second, the Database's "home" has migrated from Washington University and, starting with the current (Oct. Term 2024), will be archived at Penn State University (click here).

Data on Homicide Plea Bargaining in Wisconsin

While various criminal law law theories and issues are increasingly subjected to data, critical wrinkles remain largely "hidden in plain sight." One key issue involves how prosecutors engage in--and deploy--prosecutorial discretion. A deeper and more granular understanding of how prosecutorial discretion cashes out when it comes to plea bargaining remains both pressing and enormously consequential.

Unsurprisingly, data limitations pose one critical barrier to studies on how plea bargaining actually works on the ground. One recent and helpful contribution, Plea Bargaining in Homicide Cases: An Empirical Exploration in One State, involves data on one crime type (homicide), from one state (Wisconsin), between 2005-2018.

Under Wisconsin law, defendants alleged to have committed the most serious homicides can be charged with a Class A felony. Homicide defendants charged with a "lesser" crime, a Class B felony, confront a comparatively lighter sentence. As the authors, Michael O’Hear (Marquette) & Darren Wheelock (Marquette—social & cultural sciences), note, "one common plea deal is for the prosecutor to reduce the charge from a Class A to a Class B homicide in return for the defendant’s surrender of his or her right to a trial." And their paper explores "which Class A defendants get this big break [a Class B felony charge] and why." To do so, the authors examine a set of a 293 cases where the defendant was initially charged with a Class A homicide and then pled guilty. In 198 of these cases, the defendant was permitted to plead guilty to a reduced Class B charge. In the remaining 95 cases, the defendant confronted the original Class A charge (despite having plead out).

Their findings include some possible surprises. One includes the apparent lack of statistical salience of defendant or victim race. Factors that did achieve statistical significance, however, include whether a defendant was defended by a court-appointed lawyer. The paper's abstract follows.

“Even the most serious criminal cases are often resolved through plea bargaining, with potentially dramatic consequences for the sentence ultimately imposed. In Wisconsin, the most serious type of homicide is a Class A felony, which results in a mandatory life sentence. However, many individuals who are initially charged with the Class A offense are given the opportunity to plead guilty to a lesser Class B offense, which does not trigger any mandatory minimum sentence at all. In this Article, we compare the characteristics of the cases in which the defendant pleads guilty to a Class A offense (a relatively harsh outcome) with the characteristics of the cases in which the defendant pleads guilty to a Class B offense (a relatively lenient outcome). We find statistically significant differences between these two sets of cases in such areas as county of prosecution, appointed versus private-pay defense lawyer, and gender of defendant, victim, prosecutor, and defense lawyer.”

From PACER to SCALES: Access to Federal Court Data

Even basic research questions on Art.III courts and federal litigation typically confront consequential barriers to needed data (e.g., federal court records). While the federal government's PACER platform and a few third-party operations provide some relief, both possess their own respective shortcomings and limitations (including, but not limited to, cost).

Initial fruits of a new collaborative effort based at Northwestern University, SCALES, have emerged recently. SCALES is an online platform that seeks to "assemble federal court records, systematically organize them and extract key information, and—most importantly—make them freely available to the public." What exists now includes access to information and records for all federal cases initiated in 2016 and 2017. As SCALES' mission includes "to make court records freely accessible to the public," the platform aspires to, in the future, expand coverage to include "all years."

In a recent paper, The SCALES Project: Making Federal Court Records Free, the authors, David Schwartz (Northwestern) et al., explain the SCALES initiative in greater detail. As well, the project also benefits from its own website (click here).

Initial Data on Trump Judges' Performance

Persistent general public complaints about Trump's Art.III judicial picks include that they are "unqualified" and were drawn from an unduly narrow pool of traditionally plausible candidates. Of course, and as they say, "where data are scant, 'opinions' bloom." In a recent recent paper, How Different Are the Trump Judges, Stephen Choi (NYU) & Mitu Gulati (UVa) bring some initial data to these prevailing conventional wisdoms. The paper sets out to examine how Trump-appointed judges "have done compared to other presidents' appointments." Their findings challenge prevailing and widely-held assumptions.

The paper includes all active circuit court judges between January 1, 2020, through June 30, 2023. While the analyses are largely descriptive, the paper explores judge performance in terms of three key metrics: productivity, influence, and the degree to which a judge is willing to "deviate" from other judges, and particularly from "those closest to them in terms of political ideology." As the paper's abstract (below) implies, and contrary to public predictions of underperformance, the authors find that "Trump judges outperform other judges."

"Donald J. Trump’s presidency broke the mold in many ways, including how to think about judicial appointments. Unlike other recent presidents, Trump was open about how 'his' judges could be depended on to rule in particular ways on key issues important to voters he was courting (e.g., on issues such as guns, religion, and abortion). Other factors such as age and personal loyalty to Trump seemed important criteria. With selection criteria such as these, one might expect that Trump would select from a smaller pool of candidates than other presidents. Given the smaller pool and deviation from traditional norms of picking 'good' judges, we were curious about how the Trump judges performed on a basic set of measures of judging. One prediction is that Trumpian constraints on judicial selection produced a different set of judges that underperform compared to judges appointed by other presidents. Using data on active federal appeals court judges from January 1, 2020 to June 30, 2023, we examine data on judges across three different measures: opinion production, influence (measured by citations), and independence or what we refer to as “maverick” behavior. Contrary to the prediction of underperformance, Trump judges outperform other judges, with the very top rankings of judges predominantly filled by Trump judges."

High Court Data From 'Down Under'

Thanks to Zoe Robinson (ANU) and Patrick Leslie (ANU), a database of all cases decided by the High Court of Australia between 1995 and 2020, is now publicly-accessible (click here). The database was consciously designed to supply data on cases heard by the High Court of Australia as well as the justices that hear the cases and facilitate empirical analyses.

To this end, the database contains "over 315 variables for every case argued before a panel of at least two justices on the Australian High Court between 1995 and 2020, as well as variables on key identity information about the justices hearing the cases. Examples include justice votes, justice coalitions, legal issues in the case, the parties to the case, justices who granted special leave to appeal, justice gender, and justice ideology, amongst many others." Those intrigued might want to skim the dataset's codebook to get a more granular feel for what's included in the data.

Finally, and notably, the researchers strategically constructed the High Court database with an eye towards facilitating comparative work with the U.S. Supreme Court database in terms of variable categorization and naming.

Federal Court Judicial Opinons and Judge Gender

For years (decades, even), legal and political science literatures have considered whether judge gender informs judicial outcomes. While the standard interpretation emphasizes that judicial attributes (including judge gender) play a causal role, an alternative explanation considers the possibility that case characteristics (and therefore potentially outcome) become associated with judicial attributes via some other mechanism, such as case assignment, or decisions about what cases to publish or report.

Results in a recent paper, Gendered Judicial Opinions, by Michael Livermore (UVa) et al., substantially support an assignment/selection pathway explanation. To be sure, the paper is appropriately cautious cautious and notes that while its findings support the assignment/selection pathway, the possibility of a gender causation pathway cannot be properly excluded as the two are not mutually exclusive.

The paper levers a rich data set that includes all federal circuit court opinions, published and unpublished, from 2001 through 2017, except for decisions of the Court of Appeals for the Federal Circuit, en banc opinions, and opinions that include fewer than 900 characters. The authors then merged textual data from judicial opinions with judge-level demographic information collected by the Federal Judicial Center (FJC). The paper's abstract follows.

“In this paper we investigate whether gender is associated with the content of judicial opinions in the U.S. courts of appeals. Using a topic model analysis, we find that gender is a significant predictor of the content of judicial opinions. Two causal pathways could explain this result: (1) men and women judges write differently about the same cases; (2) men and women judges write about different cases, either due to assignment or selection effects. To untangle these two pathways, we carry out three additional analyses. First, we examine whether the United States as a party is associated with judge gender. We next examine whether case codes are associated with judge gender. Finally, we examine the relationship between topic prevalence and gender, controlling for case codes. Our findings lend greater support to the second pathway than the first. This result raises the prospect that prior work on gender-based differences in judicial behavior may be confounded by assignment or selection effects. Our results also raise normative concerns about gender disparities in voice and influence in the U.S. courts.”

Initial Data on Data Breach Litigation

News reports about massive data breaches are quickly becoming all-too common. While states have adopted data breach notification statutes and an increasing number of privacy statutes, litigation in this space remains largely by common law doctrines. Despite steady increases in litigation over time, quite little is know, systematically, about how such litigation has unfolded.

At the risk of stacking "turtles all the way down," a recent paper, Unto the (Data) Breach, sets out to gather initial data on data breach litigation. Drawing from WestLaw and PACER documents, the paper identifies 225 cases (between 2005 and 2022). While the analyses are exclusively descriptive, this is representative of helpful granular foundational work from which more technically sophisticated analyses can flow in the future. One take-away from the paper is that a comparative absence of a "cohesive data breach common law" likely exerts upward pressure on settlement activity. An excerpted abstract follows.

“Since the early 2000s, U.S. courts have begun hearing “data breach” liability cases, the inevitable result of a growing internet-connected technology infrastructure. The relatively recent development of case law signals a body of law in development, stunted by significant limiting factors that prevent the coalescence of legal principles. … This descriptive empirical study analyzes, in detail, 225 data breach cases from 2005-2022, reviewing these cases over a three-year period to descriptively identify key trends and changes within a case’s life on the docket.

This study identifies the type of plaintiff, settlement amounts, type and disposition of information compromised, claims, common motions, and key strategies most likely to result in a favorable plaintiff outcome. It also explores broad trends in data breach litigation, including acceptance of claims by courts, the status of future harms in 12(b)(1) and 12(b)(6) standing challenges, and the degree to which courts are willing to let cases proceed beyond preliminary motions….”

Data on U.S. Solicitor General's "Flips"

While many criticize the Supreme Court for "unsettling" Court precedent, exceptionally few focus on instances where the Solicitor General's Office ("SG") "advances legal arguments that differ from those advanced by government lawyers in previous cases." Notably, '“[e]ven those observers who defend the SG, including veterans of the office, caution that inconsistency in legal argument poses a threat to the SG’s credibility with the Court." A recent paper by Margaret Lemos (Duke) & Deborah Widiss (Indiana), The Solicitor General, Consistency, and Credibility, sets out to empirically better understand the circumstances that lead the SG to "change its position on the meaning of the law, and to unpack the connections between consistency and credibility.”

To do so, the paper leverages an original dataset that includes 131 cases drawn from 1892 to the Court's 2022 term. The authors make clear that the sample is under-inclusive in that it contains only those SG "flips" that are "evident": that is, where "the government itself, other parties or amici, or the justices flagged that the government had changed its legal position." In about 80% of the cases, the SG itself acknowledged the change in its briefing. Of course, that also means that in 20% of the cases the government "either did not discuss, or denied, the changed position."

While the paper's analyses are strictly descriptive, main findings include that changes in the SG's litigating positions have increased over time. The authors also emphasize that SG reversals (or "flips") happen for an array of (often overlapping) reasons, "many of which stem from the SG’s unique role in coordinating litigation across a vast and constantly changing federal government." An excerpted abstract follows.

“This Article offers the first comprehensive look at cases in which the Solicitor General (SG) rejects a legal argument offered on behalf of the United States in prior litigation. Such reversals have received considerable attention in recent years, as shifts in presidential administrations have produced multiple high-profile “flip-flops”—as the justices sometimes call them—by the SG. Even those observers who defend the SG, including veterans of the office, caution that inconsistency in legal argument poses a threat to the SG’s credibility with the Court. Our goal is to better understand the circumstances that lead the SG to change its position on the meaning of the law, and to unpack the connections between consistency and credibility.

… Indeed, our study calls into question the idea that ideological swings associated with changes of presidential administrations can be isolated, either in theory or in practice, from other sorts of legal, social, and technological changes that shape the government’s understanding of the law. It also shows that the connection between consistency and credibility, while intuitive at first blush, rests on a formalist understanding of law and an unpersuasive equation of the judiciary and the executive.”

Circuit Panel Effects Over Time

Most empirical judicial decisionmaking scholarship on federal circuit courts tries to exploit something close to quasi-random assignments to three-judge appellate panels. While attention typically focuses on judge votes, in a recent paper, Partisan Panel Composition and Reliance on Earlier Opinions in the Circuit Courts, Stuart Minor Benjamin (Duke) et al. instead focus on whether panel composition (partisanship) influences how judges rely on prior case law.

Drawing on a data set that includes "all published and unpublished federal appellate opinions in the Lexis database that were issued between 1974 and 2017," the authors find that circuit panel composition does influence which cases are relied upon and followed and that the influence is asymmetric. Specifically, they find that evidence of a partisan influence in this context is: 1) relatively recent; 2) most salient for circuit opinions written by 3-Democrat panels; and 3) more likely in relatively "high-profile" cases. An excerpted abstract follows.

"Does the partisan composition of three-judge panels affect how earlier opinions are treated and thus how the law develops? Using a novel data set of Shepard’s treatments for all cases decided in the U.S. courts of appeals from 1974 to 2017, we investigate three different versions of this question. … We find that partisanship does, in fact, structure whether earlier opinions are followed and that these partisan effects have grown over time—particularly within the subset of cases that we believe are most likely to be ideologically salient. Since legal doctrine is developed by building upon or diminishing past opinions, these results have important implications for our understanding of the development of the law."

The Bruen Decision After One Year: An Empirical Look

In its 2022 Bruen decision, the Court ruled (6-3) that a New York law requiring that applicants “show cause” in their conceal carry firearm license applications unconstitutionally infringed on Second Amendment protections.

Since the Bruen decision, a stream of additional lawsuits challenging various other state and federal firearm regulations has been decided. Assessing these post- Bruen court decisions may hold some clues about the Bruen decision’s impact on the Second Amendment landscape.

To this end, in One Year Post-Bruen: An Empirical Assessment, Eric Ruben (SMU) et al., analyze more than 450 claims, flowing from 326 cases, decided after Bruen. The authors note that the pace (and volume) of litigation dramatically increased after Bruen as well as the lawsuits’ efficacy. The authors also observe that Bruen’s analytic framework “has not meaningfully constrained judges.” The paper’s abstract follows.

"In the year after New York State Rifle & Pistol Association v. Bruen, a steady stream of highly publicized opinions struck down a wide range of previously upheld gun restrictions. Courts declared unconstitutional policies ranging from assault weapon bans to domestic abuser prohibitions to various limits on publicly carrying handguns. Those opinions can frequently be paired with others reaching the opposite conclusion. The extent to which Bruen shook up the Second Amendment landscape and has caused widespread confusion in the courts is starting to come into focus.

This Article measures Bruen’s after effects by statistically analyzing a year’s worth of Second Amendment opinions. We coded more than 450 challenges for dozens of variables including both case and judge characteristics, resulting in a comprehensive post-Bruen Second Amendment dataset. The findings of our analysis provide an objective basis for assessing the upheaval wrought by Bruen and highlight both unanswered questions and immense challenges for Second Amendment doctrine in the coming years."

New Thinking About an Old Problem: Selection Effects

Keith Hylton (Boston Univ.) and Sanghoon Kim (Buffalo--econ.) distill their recent paper's challenge nicely. When it comes to much of empirical work on legal (and court) outcomes, the full potential of multiple regression "has not been realized, largely because of one reason: the information contained in court opinions comes from a selected sample."

For all the obvious reasons, much of empirical legal research exploits court outcomes as data. One challenge, however, is that the comparatively smaller subset of formal legal disputes that persist to a judicial outcome almost assuredly systematically differs from the much larger subset of legal disputes that privately settle. My former mentor and colleague, Ted Eisenberg, frequently reminded me that "one cannot over-state the importance of selection effects in empirical legal research."

While the challenge endures, solutions are not entirely obvious. In Trial Selection and Estimating Damages Equations, Hylton and Kim develop an interesting suggestion. Specifically, they articulate a structural econometric model that, they argue, better accounts for litigant selections in the appellate context. The paper's abstract follows.

"Many studies have employed regression analysis with data drawn from court opinions. For example, an analyst might use regression analysis to determine the factors that explain the size of damages awards or the factors that determine the probability that the plaintiff will prevail at trial or on appeal. However, the full potential of multiple regression analysis in legal research has not been realized, largely because of the sample selection problem. We propose a method for controlling for sample selection bias using data from court opinions."

Federal Judicial Data: Greater Transparency or Opacity?

While not itself an empirical paper per se, The Necessary and Proper Stewardship of Judicial Data, by Zachary Clopton (Northwestern) and Aziz Huq (Chicago), engages in a discussion near-and-dear to most legal empiricists' heart. Specifically, their paper sets out to develop "the first comprehensive descriptive, doctrinal, and normative analysis of judicial data.”

As the authors note, judicial (especially Art. III courts) data are "valuable in governance and for those interested in understanding how they are being governed. In the absence of hurdles or walls, it can illuminate the operation of one branch of the national government. Constitutional lapses can be brought to light through retail or wholesale data." As such, data flowing from Art. III courts and institutions can promote various public goods by, in part, advancing a better understanding of, or improvements to, judicial institutions.

One critical anomaly is that, as compared to the legislative and Executive branches, judicial branch data remain "largely terra incognito for scholars and the general public alike. Instead, public attention rushes to high-profile cases of leaked Supreme Court opinions and private influence campaigns.” While why this is so remains unclear (to me, anyway), that it is so, however, is obvious to any empirical legal scholar who labors and writes in this space. And the stakes incident to the "transparency v. opacity" debate grow more acute as new scholarly tools emerge (e.g., natural language processessing, text-scrapping).

As the authors make clear, under current policy and practice judicial data "might be lost, dissipating into air, or kept under wraps, never yielding up their secrets." In contrast, a commitment to greater transparency invites characterizing judicial data as a "new public asset that can then be tapped to create public goods for the benefit of all."

Legal Precedent and Decay

While the idea of legal precedent implies some degree of durability, precedents are not immortal. Over time, they lose their capacity to influence action; eventually they cease to be of anything other than historical interest. Although precedential decay's existence is generally accepted, the determinants of precedential decay are comparatively under-studied.

Yun-chien Chang (Cornell) & Geoffrey Miller’s (NYU) recent paper, Decay of Precedent in State Supreme Courts, extends this literature by analyzing citation decay patterns in state supreme court opinions, drawing on three different (and large) data bases. Their paper examines both “backward-looking” precedential decay—the frequency with which opinions in a given year cite to opinions from prior years as well as “forward-looking” decay—the frequency with which specific precedents are cited in later opinions. One key finding includes the extraordinary persistence of exponential decay across all of the data. The paper’s abstract follows.

“This paper investigates the decay of precedents in three unique databases: citations in all published opinions by state supreme courts in 2019 (102,555 citations); citations in all published opinions by state supreme courts in 2003 (114,777 citations); and citations by all state and federal courts to state supreme court opinions issued in 2003 (259,627 citations). We find that the frequency of citation drops off by a roughly constant ratio with each passing year. The large scale of our yearly data and the time studied (1904–2019) suggests that this pattern must be the result of underlying forces not having to do with particular circumstances of time or place. We also find that cases with dissents tend to cite older cases and longer decisions tend to be cited more often and faster. Criminal law cases, as compared to constitutional law cases, tend to cite more recent precedents, while trust and estate cases tend to cite older precedents. We offer some conjectures about possible reasons for the observed differences in decay rates.”

SCOTUS Nominations, By the Nimbers and Over Time

Charles Cameron (Princeton--politics ) & Jonathan Kastellec (Princeton--politics) recently published a book, Making the Supreme Court (OUP), that tracks 90 years of POTUS appointments to the Supreme Court. The Book's description (below) makes clear what the authors attempt to achieve with their book. The authors' execution benefits from data, including on:

• Party and presidential interest in the court, including the use of “litmus tests” for Supreme Court nominees;
• The type of selection processes employed by the president;
• The background characteristics of Supreme Court nominees, and those who made the president’s “short list”;
• Interest group participation in Supreme Court confirmation battles;
• Media coverage of Supreme Court nominees;
• Public opinion of nominees;
• Senate roll call voting on Supreme Court nominees; and
• Simulations on the future ideological trajectory of the Supreme Court.

But what particularly caught my eye was that the authors make available the data (and related coding) that they used in their analyses (click here).

“Appointments to the United States Supreme Court are now central events in American political life. Every vacancy unleashes a bitter struggle between Republicans and Democrats over nominees; and once the seat is filled, new justices typically vote in predictable ways. Making the Supreme Court examines 90 years of American political history to show how the growth of federal judicial power from the 1930s onward inspired presidents, the political parties, and interest groups to shape judicial policy through appointments. The result is a new politics aimed squarely at selecting and placing judicial ideologues on the Court, which has transformed the Court into an ideologically driven and polarized branch. Based on rich data and qualitative evidence, Making the Supreme Court provides a sharp lens on the social and political transformations that created a new American politics.”

Lower Fedreal Courts and Religious Claims: An Update

The Supreme Court's ongoing readjustment of long-standing Free Exercise and Establishment clause tensions has renewed scholarly attention to courts' resolutions of religious claims. And in a recent paper, Trump's Lower-Court Judges and Religion: An Initial Appraisal, Stephen Choi (NYU) et al. contribute to a growing empirical literature by levering data on Trump-appointed Circuit Court judges in Free Exercise Clause cases and comparing Trump-appointed judge outcomes with those of Non-Trump Republican- and Democratic-appointed judges.

What the authors find includes that "Trump’s judicial appointments are more religious but no less qualified than other judges," and that "Trump judges are more likely to vote in favor of the plaintiff than not only Democratic judges, but also judges appointed by other Republican presidents."

Notably, however, while Trump-appointed judges vote “pro-religion” (that is, in favor of a religious plaintiff asserting a free exercise claim) more frequently than other judges, this tilt distributes unevenly across various religions. For example, and as the table below illustrates, the authors find “... different voting patterns for Islam. For cases involving Islam, Trump judges vote in favor of Muslim plaintiffs in only 19.2% of the cases compared with 34.5% for Non-Trump Republicans and 48.0% for Democrats.” The authors note that the overall pattern that emerges is “consistent with conventional wisdom: Democrats tend to protect minority religions, and Republicans tend to protect Christianity (and possibly Judaism).”

 

 

The paper's abstract follows.

“It is widely believed that President Donald Trump’s judicial appointments reflected a strategy of appeasing evangelical Christians and other religious groups that favored a more conservative, Christian judiciary, and that in pursuing this strategy Trump sacrificed quality. We explore evidence for this theory by examining the biographies and credentials of Trump’s lower court nominees and the voting records of his judicial appointments to the circuit courts in free exercise cases. We find that Trump’s appointments to the lower courts have stronger or more numerous religious affiliations, and are affiliated with the Federalist Society and the National Rifle Association at unusually high rates, but are no less well credentialed than other judges are. We also find that Trump’s appointments to the circuit court more frequently vote in favor of Christian plaintiffs, and less frequently vote in favor of Muslim plaintiffs, in free exercise cases than judges appointed by other Republican presidents and by Democratic presidents.”

Data on Local Prosecutor Campaign Contributions & Electoral Outcomes

Building on prior descriptive work (here) that focuses on local prosecutor elections in the U.S., in a recent paper, Donating to the District Attorney, Carissa Hessick (UNC) et al., broaden their focus to assess how district attorney campaign contributions map onto electoral success. To inform the authors' interest in "the local politics of mass incarceration," the paper exploits a unique data set that links contribution and election results data. Merging these two data sets allows one to know whether a given recipient [local prosecutor candidate] was an "incumbent or challenger, the recipient’s vote share in both the primary and general election, and whether the election was contested in the primary election, the general election, or either election."

Such data provide a "more nuanced picture of prosecutorial politics, allowing us to make important distinctions between offices and interest groups, suggest new accountability measures, and better evaluate how elections can lead to reform." What the authors find propels them to conclude that "accountability mechanisms beyond elections are necessary to ensure prosecutors do not exercise their significant discretionary power unchecked." The paper's excerpted abstract follows.

“… The study offers a more complete empirical account of prosecutor accountability by analyzing contributions to local candidates as well as their election results. It details the amount of money in local prosecutor elections, including from interest groups, and the relationship between candidate fundraising and success. The stark differences across the country underscore that the more than two thousand local prosecutors are not a monolith; some offices are best understood as political, with contested elections and significant amounts of campaigning, while most appear more bureaucratic...”

Judicial Decisionmaking: Moving Beyond Case Outcomes

While conceding that "judges matter" as a general matter, a recent paper, Do Judges Matter?, considers a slightly narrower question: what about judges matters when it comes to the more mundane aspects incident to the business of judging?" More specifically, the authors, Yun-chien Chang (Cornell) and Geoffrey Miller (NYU), explore potential relationships "between a judge’s personal characteristics and the day-to-day business of courts – the writing of opinions, the use of citations, the frequency of dissent, and like factors."

Exploiting a hand-collected data set of 4,591 substantive decisions issued by 48 state supreme courts in 2003 (excluding Alaska and Hawaii), the paper finds that while state supreme courts themselves vary in how they carry out their judicial business (e.g., citation distributions, frequency of dissents), personal characteristics of judges, by contrast, have only a "negligible impact." The abstract follows.

"An extensive literature examines whether characteristics of judges correlate with votes on cases. These studies generally consider the judges’ votes on the merits of cases. Examining a data set of 4,591 decisions issued by 48 state supreme courts in 2003, we consider whether judges’ personal features affect their opinion writing. We find virtually no significant differences along any of the dimensions under review. Judicial characteristics matter only indirectly — ideological differences increase dissensus in public-law cases, which increases opinion length and citation numbers. Our study suggests that when carrying out the ordinary business of courts, judges are almost all the same."

Data on the Fourth Amendment's "Good Faaith" Exception's Application

While the Fourth Amendment is designed to protect criminal defendants against prosecutors' use of unlawfully obtained evidence, an important exception exists for evidence obtained by law enforcement officers who rely in good faith on existing legal authority. Thus, if police rely on a statute that turns out to be unconstitutional or invalid warrants or precedent, the evidence obtained is nonetheless admissible under the good faith exception.

Empirical analyses on how the good faith exception works "on the ground," however, are scant. To this end, a recent paper, The Reality of the Good Faith Exception, by Matthew Tokson (Utah) and Michael Gentithes (Akron), helps to fill a void.

Their paper draws from available published (and unpublished) judicial decisions (N=5,531) that mention the Fourth Amendment or "motion to suppress" across three separate months (July 2015, 2018, and 2021). Among the paper's findings includes a discernible uptick in the exceptions salience over time (see Figure 1, below).

Figure 1: Good Faith Exception (GFE) Prevalence Over Time - Proportion of Fourth Amendment Suppression Cases Involving the Good Faith Exception

The authors are correctly mindful that selection effects certainly influence their descriptive findings. The authors argue that selection effects in this context likely lead, on balance, to an under-counting of suppression hearings to the extent that "defendants and their attorneys likely decline to file suppression motions because of the good faith exception." (emphasis added)

Overall, the paper argues that "the [good faith] exception, originally confined to narrow circumstances, is turning into a protection for nearly any investigatory activity with an arguable connection to an existing precedent or statute" and that "[c]ourts rely on the exception with remarkable frequency, and a large proportion of courts invoking the exception avoid any substantive constitutional ruling, stunting the development of Fourth Amendment doctrine."

Lay Understandings of Civil Settlements

When it comes to resolving formal civil disputes, popular imagination, partly fueled by the media and entertainment industry, is dominated by "courtroom drama" and jury trials. Empirical work over the decades, however, paints quite a different picture, and one dominated by the "disappearing trial." Thus, as Jennifer Robbennolt (Illinois) et al. make clear in Settlement Schemas: How Laypeople Understand Civil Settlement, "for all the attention paid to the real and imagined drama of the courtroom, these depictions overlook the reality of how many disputes are resolved: through any means but trial."

In the service of seeking a better understanding of how the public understands civil settlements, the authors surveyed a representative sample of more than 1,000 U.S. adults. Key findings include that while the public conveyed a widespread general understanding about civil settlements, evidence of misconceptions about the details, including the role of a jury and settlement scope, also emerged. The paper's abstract follows.

"What does the public think it means to ‘settle’ a civil case? Most legal disputes in the United States end in an agreement to settle, but little is known about what laypeople think about settlement. To fill this gap, we took a direct approach: we asked a nationally representative sample of U.S. adults – more than one thousand of them – basic questions about settlement. We found widespread understanding about the essential nature and frequency of settlement, but persistent, though not universal, misconceptions about the details, including the role of a jury and settlement scope. Because settlement is such a pervasive part of the U.S. legal system, the system’s legitimacy turns in part on how the public understands and views civil settlement. The survey reported here provides a foundational study of the understandings and framework – the schemas – that the public bring to settlement."

Sentencing as a Product of Plea Agreements

When it comes to explaining criminal sentences, conventional wisdom emphasizes either judges or, given that criminal convictions largely flow from negotiated plea agreements, prosecutors. A closer look, however, reveals far more nuance and complexity and a dynamic process involving multiple parties concurrently rather than a static event dominated by one actor.

In a recent "closer look," Jeffrey Bellin (William & Mary) & Jenia Turner (SMU) explore 77 plea bargains in the first half of 2019. Noting that comprehensive empirical work on judicial responses to negotiated plea deals is sparse, In Sentencing in an Era of Plea Bargains makes a few helpful contributions. First, the authors' deep dive into 77 pleas bargains, though descriptive, revels some initial, general contours of these agreements. Notably, while judges ultimately approved all of the 77 plea bargains in the paper's sample, judges departed from the negotiated sentencing recommendation in about one-third of the cases. Also notable is that in virtually all instances where the judge departed from the negotiated sentence, she did so in a direction that favored greater leniency. Second, in separate analyses the authors examine 31 plea agreements that elicited judicial rejections. Somewhat perplexingly, judges rejected most plea agreements (74%) because they felt the recommended sentence was too lenient.

Obviously, descriptive studies that lack statistical power are limited in important ways. At the same time, however, such studies can help by pointing future researchers in potential helpful directions. The paper's (excerpted) abstract follows.

“The literature offers inconsistent answers to a question that is foundational to criminal law: Who imposes sentences? Traditional narratives place sentencing responsibility in the hands of the judge. Yet, in a country where 95 percent of criminal convictions come from guilty pleas (not trials), modern American scholars center prosecutors – who control plea terms – as the decider of punishment. This Article highlights and seeks to resolve the tension between these conflicting narratives by charting the pathways by which sentences are determined in a system dominated by plea bargains.

After reviewing the empirical literature on sentence variation, state and federal plea-bargaining rules and doctrines, and conducting some empirical analysis of our own, we conclude that neither of the competing narratives is correct. Sentencing in the United States has become a dynamic process with substantial contributions from multiple actors, not a static event controlled by any single actor.”

How Summary Judgment Changes Vary Across Subject Areas

The Supreme Court upended summary judgment doctrine with a trilogy of cases in 1986. At the time--and largely persisting into current prevailing wisdom--it was assumed that the 1986 changes made summary judgment comparatively more available and, thus, of greater benefit to defendants. While empirical evidence remains somewhat scant, emerging findings paint a far more complex and nuanced picture.

A recent contribution to this research literature comes from Jonathan Nash (Emory) and Daniel Sokol (USC). In The Summary Judgment Revolution that Wasn't, the authors lever six separate (though related) datasets that draw from federal court decisions in three discrete areas (antitrust, civil rights, and securities regulation) and exploit the 1986 trilogy with pre- and post-intervention comparisons. At bottom, the paper finds "no evidence of the trilogy influencing summary judgment decision making in securities regulation, and at best mixed evidence of such an effect in civil rights cases. In contrast, we find that the trilogy did have a strong statistically significant effect on decision making in antitrust cases, at both the district court and court of appeals levels." The abstract follows.

"The U.S. Supreme Court decided a trilogy of cases on summary judgment in 1986. Questions remain as to how much effect these cases have had on judicial decision making in terms of wins and losses for plaintiffs. Shifts in wins and losses and what cases get to decisions on the merits impact access to justice. We assemble novel datasets to examine this question empirically in three areas of law that are more likely to respond to shifts in the standard for summary judgment: antitrust, securities regulation, and civil rights. We find that the Supreme Court’s decisions had a statistically significant effect in antitrust, an ambiguous effect in civil rights cases, and no effect in securities regulation. We also find that, in the trilogy’s wake, antitrust appellate cases were far more likely to cite trilogy cases—and in particular the one trilogy case that was an antitrust case—than appellate cases in the other areas. This suggests that the lone trilogy case that arose in antitrust had an effect on decision making in that field, but that the trilogy had a limited effect across other substantive areas. This finding differs from Twombly and Iqbal where an antitrust decision ultimately reshaped the entire body of law around motions to dismiss."

Experimental Judicial Decisionmaking Studies & External Validity

For experimental judicial decisionmaking studies, the "gold-standard" research design includes using actual judges as participants. My Cornell colleague (and good friend) Jeff Rachlinski is a leading scholar in this space. For an array of reasons (including practical reasons), too much of the experimental research in this field is hampered by researchers' use of students (including law students) as the experimental participants. To the extant that researchers seek to generalize results from judicial decisionmaking studies to the "universe of judges," conventional  wisdom suggests that the use of students in these experiments raises important external validity questions.

Of course, such (and most) "conventional wisdoms" rest on empirical assumptions. And in a recent paper, Can Law Students Replace Judges in Experiments of Judicial-Making?, Holger Spamann (Harvard) and Lars Klohn (Humboldt) submit this question to an experiment. The answer to the question they pose in their paper's title?: "unfortunately, no." The paper's abstract follows.

"Experimental research on judicial decision-making is hampered by the difficulty of recruiting judges as experimental participants. Can students be used in judges’ stead? Unfortunately, no. We ran the same high-context 2×2 factorial experiment of judicial decision-making focused on legal reasoning with 31 U.S. federal judges and 91 elite U.S. law students. We obtained diametrically opposed results. Judges’ decisions were strongly associated with one factor (sympathy, i.e., bias) but not the other (law). For students, it was the other way around. Equality between the two groups is strongly rejected. Equality of document-view patterns—a proxy for thought processes—and written reasons is also strongly rejected."

Modeling Rare Events--SCOTUS Clerkships

Of the approximate 36,000 law school students who graduate each year, only 36 or so end up securing a SCOTUS clerkship. While a handful of biographical and anecdotal accounts exist, very few systematic data exist on the "1-in-1,000" clerkship lottery.

A recent paper by Tracey George (Vanderbilt), Albert Yoon (Toronto), and Mitu Gulati (Virginia), Some Are More Equal Than Others: U.S. Supreme Court Clerkship, digs into 40 years worth of data and provides rich granularity about this statistical rare event. In the main, while some unexpected twists emerge, what the authors find generally comports with perceived wisdoms. For example, a disproportionate number of clerks emerge from five elite law schools. One interesting change, however, is that while a SCOTUS clerkship once led "primarily to government and universities [law faculties]," former clerks now increasingly land at "law firms and corporations." An excerpted abstract follows.

"... Yet, we lack a clear story about who wins this clerkship lottery. This Essay seeks to provide that story. Our analysis relies on new datasets of all Court clerks who served between 1980 and 2020, including the details of their path to the Court and their road after. We amend and expand on theories of success in this important labor market. All clerks are highly qualified, yet the same is true for many not chosen. We find that educational pedigree, as opposed to academic performance or any other qualification, often distinguishes the winners from the also-rans. The Court clerkship selection process, like other elite labor markets, proves to be a blend of status and merit where status often prevails. Our analysis does not end there, however. We go on to look at where this forty-year cohort is currently working and confirm that once attained, a Court clerkship does lead to a bounty of opportunities including a return to the Court as a justice. Thus, the Court clerkship lottery is an important labor market not only to lawyers but also to society writ large. In the elite legal labor market, some people are, in fact, more equal than others."

More Evidence of a Racialized Death Penalty

Notably, race does not emerge as a central concern in the Court’s Furman decision, now more than fifty years old. In the decades since Furman, however, empirical evidence continues to mount documenting that "racial discrimination infects the administration of the death penalty." A recent paper by Jeffrey Fagan et al. (Columbia), Getting to Death: Race and the Paths of Capital Cases after Furman, represents a recent and important contribution to this literature. Researchers visited each judicial circuit in Georgia to record the details of every murder case that entered the courts from 1994–2005. Analyses of the 1,317 death-eligible murders suggest that “race continues to matter, and that the statutory aggravators that distinguish death cases from other killings are themselves racialized.” A more detailed description of the paper's core findings appears in the Abstract (below).

“Decades of research on the administration of the death penalty have recognized the persistent arbitrariness in its implementation and the racial inequality in the selection of defendants and cases for capital punishment. This Article provides new insights into the combined effects of these two constitutional challenges. We show how these features of post-Furman capital punishment operate at each stage of adjudication, from charging death-eligible cases to plea negotiations to the selection of eligible cases for execution and ultimately to the execution itself, and how their effects combine to sustain the constitutional violations first identified 50 years ago in Furman. Analyzing a dataset of 2,328 first- degree murder convictions in Georgia from 1995–2004 that produced 1,317 death eligible cases, we show that two features of these cases combine to produce a small group of persons facing execution: victim race and gender, and a set of case-specific features that are often correlated with race. We also show that these features explain which cases progress from the initial stages of charging to a death sentence, and which are removed from death eligibility at each stage through plea negotiations. Consistent with decades of death penalty research, we also show the special focus of prosecution on cases where Black defendants murder white victims. The evidence in the Georgia records suggests a regime marred less by overbreadth in its statute than capriciousness and randomness in the decision to seek death and to seek it in a racially disparate manner. These two dimensions of capital case adjudication combine to sustain the twin failures that produce the fatal lottery that is the death penalty.”

Judges' Financial Holdings, Case Outcomes, and Recusal

Fundamental to the notion of a fair trial and the rule of law is the principle that a judge shall apply the law impartially and free from the influence of any personal biases. When real or perceived conflicts of interest emerge, judges are expected to recuse themselves. Despite such an expectation, however, recent Wall Street Journal reporting revealed many cases where judges violated U.S. law and judicial ethics (28 U.S.C. Section 455) by overseeing cases that involve companies in which they or their family-owned stock. Unsurprisingly, several of the cases with conflicts have been appealed.

While Chief Justice John Roberts downplayed the significance of the Wall Street Journal reporting, characterizing the judges' failures to recuse as "isolated violations" and "unintentional oversights," empirical explorations of the potential influence of judges' financial holdings on case outcomes remain rare. To address this scholarly gap, a recent paper, Judge Financial Holdings and Case Outcomes: Evidence from Judge Financial Disclosures, examines whether there is a relation between the financial holdings of judges and the outcomes of the trials over which they preside.

To construct the judges' investment data set, Harit (Univ. Texas-Dallas—mgmt.) et al. extracted judge financial disclosure data from the Free Law Project website. Their sample includes 33,726 annual disclosure reports involving 3,250 federal judges from 2000 to 2020. The authors lever the random assignment of cases to judges and examine whether a judge's rulings are related to his or her stock ownership in one or more of the parties in the case. The paper finds robust evidence that judges rule more favorably and take longer when deciding over conflicted cases as compared to unconflicted ones. It also notes that investors are positively surprised by these outcomes. The abstract follows.

"This paper examines the association between a judge's financial holdings and the outcome of legal cases. Using novel data on financial disclosures of judges and the civil case information of public firms across district courts in the United States between 2000 and 2021, we document multiple instances in which judges fail to recuse themselves in cases where they hold common stock. We find robust evidence that judges rule more favorably and take longer when deciding over conflicted cases as compared to unconflicted ones. We also find that investors are positively surprised by these outcomes, as reflected by stock returns to trial outcomes. Collectively, our results provide the first large-scale evidence of the relation between judges' financial holdings and case outcomes."

Civilian Complaints Against Police & Prosecutor "Upcharging"

Citizens’ engagement with police serves as a valuable resource. One particular form of citizen engagement—civil complaints—can impose a serious cost to the targeted police officer as well as the police department. To try to reduce the potential negative effects of a citizen complaint against law enforcement agents, other criminal justice actors could act to re-frame the encounter such that the optics look unfavorably on the complaining citizen. Since a defendant cannot seek monetary damages in civil court unless his/her conviction or sentence was declared invalid, prosecutors may increase the number or the severity of charges that the defendant faces to propel the defendant toward accepting a plea deal, which would prevent the defendant from seeking further financial reparations. In a new paper, Do Civilian Complaints Against Police Get Punished?, DeAngelo (Claremont Colleges) et al. pursue this question empirically.

To do so, the authors examine arrest, complaint and charging data from Cook County (Illinois). The results indicate that a positive, causal link between citizen complaints and an increased number of total charges filed against a defendant exists. The paper also notes that a civilian complaint yields a larger plea discount, likely to induce a plea agreement. Notably, the main findings are entirely driven by charges filed directly by the prosecutor’s office rather than directly by the police. Given that prosecutors are likely filing charges after civilian complaints have been filed against law enforcement, it appears that prosecutor behavior is leading to upcharging in response to civilian complaints and not police decisions. An excerpted abstract follows.

“… This paper empirically investigates one way in which criminal justice institutions respond to civilian complaints. Namely, criminal prosecutors can upcharge a defendant who files a civil complaint against law enforcement. By upcharging, the prosecutor can increase the likelihood that a defendant will accept a plea deal, thus preventing the defendant from seeking monetary damages in civil court (Heck vs. Humphrey, 1994). Using data on citizen complaints and criminal charge outcomes from Cook County (Illinois), we find a strong causal link between a citizen filing a complaint and the total number of charges filed.”

Theory's Role in ELS

A not infrequent, though diminishing, canard is that "too many" empirical legal studies papers are "atheoretical." While the claim itself is an empirical one (and one that I side-step), a larger, broader point endures. Specifically, the best of ELS is certainly informed by well-developed theory. In a recent paper, The Role of Theory in Empirical Legal Studies, Tonja Jacobi (Emory) makes this case in a brief and widely accessible paper. While perhaps targeted to "early career" ELS scholars and non-technical consumers of empirical research, the paper helpfully reminds all of theory's essential role in empirical legal studies. The paper's abstract follows.

"The growing field of empirical legal studies is not an alternative to theory: within every empirical inquiry is an implicit formal theoretical model. Further, good empirical tests should be inspired by substantive theories of the phenomena being analyzed. This essay explains how theory is built into all empirical testing and illustrates why empirical tests are best informed by well-developed theories. It uses examples of empirical studies of both US Supreme Court and Australian High Court oral argument to illustrate how these issues apply to the central issues at the intersection of empirical legal studies and constitutional law: the role of judicial ideology and strategy, and predicting judicial behavior. The essay explores common errors in both conducting and consuming empirical work, and presents some solutions, including both methodological fixes and how to draw on substantive expertise. It aims to help the non-technical reader understand when counterintuitive results should be queried and when, in contrast, resistance is more reflective of priors of the reader rather than potential weaknesses of the work. And it acts as a reminder to technical readers and empirical practitioners that theory is built into every empirical test, and so must be taken seriously."

Case Outcomes & Judge Financial Conflicts

To assist with insuring that "justice is blind," a presiding judge is expected to recuse from any case where he/she is financially conflicted with any of the litigating parties. Evidence that this expectation is not being consistently met was widely reported in 2020-2021. Indeed, the public controversy elicited a response from Chief Justice John Roberts. In his 2021 annual report on the federal judiciary, and in direct response to the controversy surrounding numerous federal judges, Chief Justice Roberts downplayed the controversy’s significance, characterizing the incidents as "isolated violations" and "unintentional oversights."

Notwithstanding Justice Roberts' characterizations, given the widespread implications posed by judges' failures to recuse when proper it is important to understand whether these recent episodes are one-time incidents or point to a larger, systemic problem. To address this question, in a recent paper, Judge Financial Holdings and Case Outcomes: Evidence from Judge Financial Disclosures, Tuhin Harit (Univ. Dallas--Mgmt.) et al. examine whether there is a relation between judges' disclosed financial holdings and the outcomes of the trials over which they presided.

Drawing from a sample that includes cases from 2000–2021, the paper finds that “stock ownership by judges significantly improves the chances of a litigant's winning a case.” In cases where the presiding judge is financially conflicted with either the defendant or plaintiff, “their chance of a favorable decision improves by 3.3% and 4.3%, respectively, after controlling for judge, industry, district, case file year, and firm characteristics.” An excerpted abstract follows.

"... Using novel data on financial disclosures of judges and the civil case information of public firms across district courts in the United States between 2000 and 2021, we document multiple instances in which judges fail to recuse themselves in cases where they hold common stock. We find robust evidence that judges rule more favorably and take longer when deciding over conflicted cases as compared to unconflicted ones. We also find that investors are positively surprised by these outcomes, as reflected by stock returns to trial outcomes. Collectively, our results provide the first large-scale evidence of the relation between judges' financial holdings and case outcomes.”

The Empirics of Choice of Law

In an effort to peer into judicial decisionmaking more generally, a recent paper by Dan Klerman (USC), Bias in Choice of Law: New Empirical and Experimental Evidence, levers both observational and experimental data drawn from the vehicle accident context to examine choice of law disputes. The paper considers three leading sources of choice of law bias: pro-resident, pro-plaintiff, and forum law. The findings intrigue and complicate prevailing conventional wisdoms.

Specifically, core findings include that while judges seem to favor the plaintiff, judge decisions do not tilt in favor of application of the law they are most familiar with (the law of the forum). While state court judges favor local residents, federal court judges do not. Although the traditional “place of the injury” choice-of-law rule was more constraining than modern American choice-of-law standards (such as the Restatement Second), even judges under the traditional rule favored the plaintiff more often than seemed warranted. The abstract follows.

"Scholars have argued that modern American choice-of-law is subject to three biases: (1) a bias in favor of residents of the forum state (pro-resident bias), (2) a bias in favor of plaintiffs (pro-plaintiff bias), and (3) a bias in favor of the law of the forum (forum law bias). This article brings new evidence about these biases from a comprehensive database of vehicular accident cases and from the first experiment related to choice of law. Overall, there is some evidence of a pro-resident bias in state (but not federal) court, some evidence of pro-plaintiff bias in both federal and state court, and no evidence of a pro-forum bias."

Business Litigants and the Roberts Court

Assessing “How pro business is the Roberts Court?” is both straightforward and complicated. Based on how business litigants fare in the Roberts Court versus non-business litigants over time, business does quite well. The win rate for business in the Roberts Court, 63.4 percent, is 15 percentage points higher than the next highest rate of business wins over the past century (the Rehnquist Court, at 48.3 percent). While it may be tempting for many to conclude that the current pro business tilt is purely a function of a Republican majority of justices on the Roberts Court, as Lee Epstein (USC) and Mitu Gulati (Virginia) argue in a recent paper, A Century of Business in the Supreme Court, 1920-2020, “this is only the beginning of the story.”

Epstein and Gulati bring data from the Supreme Court Database to the question. Starting with the 11,121 orally argued cases resulting in a signed opinion or judgment between the 1920-2020 terms, their paper focuses only on those 4,070 cases (35,173 votes) in which business was the petitioner (appellant) or respondent (appellee). That is, they excluded cases in which business was on both sides of a legal dispute.

Their core findings defy conventional wisdom and inject important complicating wrinkles. What they find includes: "(a) It is not just the Republicans on the Roberts Court who are more pro business than in prior Courts, but the Democrats as well; (b) The Government, through the SG’s office and across both Democratic and Republican administrations, has been much more supportive of business positions than in prior eras; and (c) An elite Supreme Court bar has emerged in recent years and businesses have hired them disproportionately so as to better influence the Court."

Empirical Contours of the Political Question Doctrine

Amid the enduring and varied debates about the political question doctrine, motivated by the Supreme Court's 1962 Baker v. Carr decision, the scholarly commentary shares one key attribute. Specifically, it dwells on the Supreme Court. Such a focus is ironic as while the Court has only rarely applied the political question doctrine, lower federal courts, by contrast, have invoked the doctrine "as a basis for dismissal in hundreds of cases." Drawing on both qualitative and quantitative (descriptive) methods, in a recent paper, The Real Political Question Doctrine, Curtis Bradley (Chicago) and Eric Posner (Chicago) explore how the political question doctrine has operated in the lower courts since the Supreme Court's Baker decision.

The authors find that lower courts use the doctrine to "evaluate their own institutional capacity to resolve politically sensitive disputes. It is the lower courts’ more limited capacity compared to the Supreme Court that explains the lower courts’ heavier reliance on the doctrine." More specifically, "the political question doctrine is a screening mechanism that the lower courts use to take account of limits on their institutional capacity. The Supreme Court has less need for this mechanism, so most of the doctrine’s life must be found in the foothills rather than at the summit of Mount Olympus. From this standpoint, the doctrine is not moribund but full of life. It is not limited in the ways suggested by the 'classical' or 'functional' theories. The lower court practice is best captured by Bickel’s prudential theory." An excerpted abstract follows.

"... We provide the first empirical account of how the doctrine has operated in the lower courts since Baker. Our account is based on both a quantitative analysis of a sample of these decisions and on a qualitative review of these and other decisions. This account reveals a political question doctrine that is substantially different from the one described in most scholarship: it is more vibrant, heavily focused on foreign affairs, often applied in non-constitutional cases, more prudential, and not a permanent disallowance of judicial review...."

Algos and Parole Decisions

Many--if not most--parole decisions remain fraught with peril. As parole boards are tasked with, to some degree, "guessing into the future," some form of risk assessment has been used to inform parole decisions since the 1920s. Despite this legacy, a recent and growing "decarceration" push has heightened public and scholarly attention to criminal risk prediction instruments. Recently, New York state amended its parole law to "center [parole board] decisions on individual’s rehabilitation in prison and future risk if released." Despite this shift in law, critics continue to argue that parole boards, such as New York's, "continue to incarcerate individuals well beyond their minimum sentence, not out of concern for public safety, but because of the nature and severity of the offense that had sent them to prison."

While concerns grow, few empirical estimates of the issue exist. In a recent paper, An Algorithmic Assessment of Parole Decisions, the authors, Hannah Laqueur (UC Davis--medicine) and Ryan Copus (Univ. Mizzou--KC), endeavor to do just that and test a risk-prediction algorithm. To do so, the paper paper exploits New York parole hearings’ data from 2012-2018. The data include individual sex, race/ethnicity, commitment crime, housing facility, parole board interview type, and the interview decision. The authors also obtained criminal history records for all individuals who had a recorded parole hearing from 2012 through 2018. The authors' risk prediction algorithm trained on the 4,168 individuals who were released on parole between 2012 and 2015 (a total of 19,713 individuals had parole hearings during this period). The primary outcome variables include "any recorded arrest within three years post-release and any violent felony arrest within three years post-release."

The authors find that the New York parole board "could have granted parole more than twice as often without increasing either the overall or violent arrest rate, or it could have released the same number of people while approximately halving both overall and violent arrest rates. Further, we find that they could have achieved these gains while simultaneously eliminating racial disparities in release rates.” The authors take care to note that they "cannot determine whether the Board is simply not as good at determining risk as our risk prediction algorithm, or is simply prioritizing non-risk related factors such as the commitment offense." The paper's abstract follows.

"Objectives: Parole is an important mechanism for alleviating the extraordinary social and financial costs of mass incarceration. Yet parole boards can also present a major obstacle, denying parole to low-risk inmates who could safely be released from prison. We evaluate a major parole institution, the New York State Parole Board, quantifying the costs of suboptimal decision-making.

Methods: Using ensemble Machine Learning, we predict any arrest and any violent felony arrest within three years to generate criminal risk predictions for individuals released on parole in New York from 2012–2015. We quantify the social welfare loss of the Board’s sub-optimal decisions by rank ordering inmates by their predicted risk and estimating the crime rates that could be observed with counterfactual risk-based release decisions. We also estimate the release rates that could be achieved holding arrest rates constant. We attend to the “selective labels” problem in several ways, including by testing the validity of the algorithm for individuals who were denied parole but later released after the expiration of their sentence.

Results: We conservatively estimate that the Board could have more than doubled the release rate without increasing the total or violent felony arrest rate, and that they could have achieved these gains while simultaneously eliminating racial disparities in release rates.

Conclusions: This study demonstrates the use of algorithms for evaluating criminal justice decision-making. Our analyses suggest that many low risk individuals are being unnecessarily incarcerated, highlighting the need for major parole reform."

Judges Without J.D.s: One First Step

In an intriguing paper, Judging Without a J.D., the authors, Sara Greene (Duke) and Kristen Renberg (Duke), emphasize an underappreciated point: In a surprising number of states lower-level state court judges do not possess the J.D. degree. Now, to be sure, even at the federal level, the J.D. degree is not required for an Art. III commission. That said, insofar as virtually every (if not every) Art.III judge today possesses a J.D. the necessary variation does not exist to support useful comparative studies.

Research not possible in the federal judicial space, however, is possible in the state judicial space. And this paper does a good job at identifying a potentially rich vein of future research possibilities. Implications from this paper's brief case study of North Carolina magistrates (where more than 80 percent do not possess the J.D. degree) hint at possible distributional concerns. And these concerns may be especially acute where non-JD magistrates intersect with pro se litigants (itself a growing trend). The paper's abstract follows.

"One of the most basic assumptions of our legal system is that when two parties face off in court, the case will be adjudicated before a judge who is trained in the law. This Essay begins by showing that empirically, the assumption that most judges have legal training does not hold true for many low-level state courts. Using data we compiled from all fifty states and the District of Columbia, we find that thirty-three states allow at least some low-level state court judges to adjudicate without a law degree, and in seventeen states, judges who adjudicate eviction cases are not required to have law degrees. Since most poor litigants are unrepresented in civil legal cases, this sets up an almost Kafkaesque scene in courtrooms across the country: Legal cases that have a profound effect on poor families, such as whether they will lose their home to eviction, are argued either in courtrooms where no one knows the law or only one party---the lawyer for the more powerful party---knows the law.

Considering data collected from a case study of North Carolina, where over 80% of magistrates do not have J.D.s, this Essay argues that allowing a system of nonlawyer judges perpetuates long-standing inequalities in our courts. It further argues that the phenomenon of lay judges is a symptom of a much larger problem in our justice system: the devaluation of the legal problems of the poor, who are disproportionately Black and Latine. This devaluation stems in part from an enduring cultural history in the United States of blaming the poor for their poverty and the associated problems of poverty. A change is in order, one that intentionally considers the expertise of judges and adopts creative solutions to incentivize specially qualified adjudicators to serve as low-level state court judges."

Federal Courts and "Certification" of State Law Questions

When federal courts need to apply state law and no clear articular of state law exists, federal judges have few options. While generating an Erie "guess" remains one option, another involves certifying the legal question to the state's highest court. Proponents suggest that state certification promotes federal-state cooperation, reduces forum shopping incentives, and, on net, reduces costs and increases efficiencies. Critics, by contrast, emphasize the unidirectional federal-state cooperation, the persistence of forum shopping, and the associated costs and delay.

While the SCOTUS and Judicial Conference have expressed growing interest in the state certification alternative of late, helpful data are scant. A recent paper by Jason Cantone (George Mason--criminology) and Carly Griffin (Univ. Ill.--ag.), Certified Questions of State Law: An Empirical Examination of Use in Three U.S. Courts of Appeals, seeks to contribute by presenting results drawn from three federal courts of appeals. While the analyses are purely descriptive, temporally bound (from 2010 through 2018), and limited to three federal circuits, they nonetheless cast some helpful initial empirical light on some of the basic contours of the certification process.

Drawing on a sample (N=218) of certified question events across three federal circuits from 2010-2018, the paper explores: "how often the certified question procedure is utilized in three U.S. Courts of Appeals, how often the circuits certify a question, how often states grant the certified question, and how long the process takes." Notably, the paper finds important variation across the three circuits when it came to state certification rates and duration. The paper's abstract follows.

"Federal courts routinely need to apply state law, but this task becomes complicated when no definitive state law exists. In these instances, federal judges have a few options, including making an Erie guess, or certifying the question of state law to the highest state court. During the U.S. Supreme Court’s October 2020 term, two opinions endorsed the certification of questions of state law procedure. However, despite recent interest by the U.S. Supreme Court and the U.S. Judicial Conference Committee on Federal-State Jurisdiction, little empirical research has examined the perceived benefits and burdens of the procedure. This Article provides new empirical data to help fill that gap.

Using a sample of 218 certified-question events across the U.S. Courts of Appeals for the Ninth, Third, and Sixth Circuits from 2010 to 2018, we examine how certified questions originate, in what types of cases, how often U.S. courts of appeal certify the questions of state law, and whether the corresponding state supreme courts grant or deny the question. Further, to address the perceived time burden associated with certification, we examine how long each part of the process takes in the appellate court, from filing to certification to termination. We found significant variation between the three U.S. courts of appeals in certification rates and timing. While we encourage future studies regarding certification of state law questions, our data can be readily used to inform courts and academic debates regarding the benefits and burdens of the procedure."

Judicial Outcomes, "Industry Peer" Stock Ownership, and Recusal

While research on "outside" influences on judicial decisions is relatively well-developed, much of that research dwells on various political influences (judge political ideology, etc.). Comparatively less well-developed is the possibility that judges' stock holdings influence their decisions in cases that involve corporate parties from "industry-peer" firms. While recusal doctrine (and practices and norms) are designed to guard against direct judicial financial conflicts of interests, indirect financial conflicts, by contrast, fall outside of formal recusal boundaries. And one example of an "indirect" financial conflict includes when a presiding judge has a financial stake in a litigant's "industry peer."

Barriers to data access are among the reasons that account for a deficit in research on this topic. In Stock Ownership of Federal Judges and its Impact on Corporations, April Knill (Fl. St.--Business) et al., explore this question after cobbling together data from an array of sources. The key move on the data front involved matching, at the individual case-level, presiding judges and their financial holdings. The authors "manually download 21,044 individual financial disclosures from JudicialWatch.org file by file... . This process results in a sample comprising 11,447 unique reports for federal judges in the years 2007 – 2012.” And within their sample, the authors report that the sample contains "observations from 69,060 federal district court cases.… Judges own industry peer stock in 8.6% of the cases in our sample.”

It is worth underscoring that in these instances the presiding judges did not violate any formal recusal obligations as the possible conflicts of interests flowing from their "industry peer" stock holdings are indirect. That said, the paper's core results imply that judges are "being influenced by their stock ownership in industry peer firms of litigants – a practice that is completely legitimate – in cases over which they preside." The paper's abstract follows.

“This paper investigates whether and how litigant peer stock ownership by federal district judges affects characteristics of case outcomes for large corporate litigants. We find that industry-peer stock ownership by district judges is associated with the following outcomes for corporate litigants named in their assigned cases: 1) an increased likelihood of judgments for the corporate litigants, 2) a decrease in the amount received by the parties suing these corporate litigants, and 3) a decrease in the length of the litigation proceedings. The random assignment of district judges to cases provides exogenous variation in the judge stock ownership. We further identify the association outlined in our base results by examining appellate court reversals of district judgments, a triple difference analysis isolating large-stake investments, and outcomes in case types that should impact industries either cooperatively or competitively. Our results survive a falsification test as well as a battery of robustness tests. Our findings underscore the importance of mandates governing judge stock ownership, and more broadly, judge conflicts of interest.”

An Empirical Assessment of What "Substantial Similiarity" Means to Courts in Copyright Litigation

Judicial assessments of copyright infringement claims remain dominated by the two-prong "substantial similarity" test. Despite this test's centrality to copyright doctrine, very little is known about how courts operationalize and apply it. In a recent paper, An Empirical Study of Copyright's Substantial Similarity Test, Clark Asay (BYU) seeks to sketch some of the test's general empirical contours. To do so, the paper, drawing from the universe of substantial similarity opinions (found on WestLaw), exploits a random sample of 1,005 opinions (district and appellate) between 1978 and 2020.

The study's key findings intrigue. First, and unsurprising to most practitioners in this space, the Second and Ninth Circuits dominate this litigation. Second, judicial analyses on the substantial similarity test tend to dwell on the second prong with comparably less judicial attention to the first. Third, judicial analyses focusing on prong two--assessing whether the defendant’s copying amounts to improper appropriation--evidence a wide diversity of approaches. As Asay notes, "[n]o dominant means exists for resolving this question, and, in any given opinion, a court typically uses multiple subtests and copyright limitations to decide this inquiry."

The paper concludes by recommending: "First, courts should maintain the two traditional prongs of the substantial similarity test as distinct and reinvigorate their assessments of similarity under the first prong of the substantial similarity test. Second, courts should make discussing and applying any relevant copyright limitations the heart of their prong two analyses to ensure that copyright law serves its constitutional purposes."

Defendant-Prosecutor Gender "Pairings" and Leniency

While various pockets on criminal law are increasingly subject to empirical analyses, one particular pocket--how prosecutors distribute discretion--is comparatively under-studied. This is so, partly, due to a comparative paucity of relevant data on the prosecutors themselves (and matching them to individual cases and dispositions). A recent paper, Gender Favoritism Among Criminal Prosecutors, by Stephanie Holmes Didwania (Wisc.), tries move the scholarly needle a bit by exploiting data on over 130,000 federal criminal defendants sentenced in 2002 through 2016 to assess possible "gender-based differences in how prosecutors exercise critical discretion."

The paper matches Sentencing Commission data and data from the Executive Office for United States Attorneys (EOUSA) which regularly publishes case data from the 93 USAOs located throughout the United States. Critically, the breadth of the LIONS data "is substantial—the data covers all cases in which a USAO was involved, including those cases that are the primary responsibility of another agency and those the USAO declines to prosecute." Despite its breath, the LIONS data set is not without important limitations including a comparatively non-trivial amount of missing data. After matching, the resultant data set data includes "131,207 federal criminal defendants sentenced between 2001 and 2016 in 63 of the 94 federal district courts."

Data set limitations aside, the paper's core finding intrigues: "That defendants receive more lenient charges when they match with their prosecutor on gender relative to when they are assigned a prosecutor of the opposite gender. Through this charging channel, prosecutor-defendant pairs that match on gender produce lower sentences relative to prosecutor-defendant pairs that mismatch." More specifically, "the leniency associated with gender-matching ultimately translates into significantly lower sentences: a defendant can expect to earn a roughly five-month shorter sentence when their prosecutor is their own gender compared to when their prosecutor is the opposite gender, which represents a roughly eight percent reduction in sentence length at the mean." The paper's abstract follows.

"Prosecutors enjoy wide discretion in the decisions they make but are largely unstudied by quantitative empirical scholars. This paper explores gender bias in prosecutorial decision-making. I find that male and female prosecutors exhibit small and statistically insignificant differences in their treatment of defendants overall but demonstrate relative leniency towards defendants of their own gender. Such favoritism at charging translates into a sentencing gap of roughly five months of incarceration for defendants who are paired with an own-gender prosecutor versus an opposite-gender prosecutor, which represents a roughly eight percent reduction in sentence length at the mean. The estimates do not appear to be driven by differences in case assignments for male and female prosecutors."

Judge Ideology and Pleadings Decisions in the Circuits

While the Twombly and Iqbal decisions separately and collectively helped galvanize empirical work in civil procedure in terms of access to federal courts, now that we're well into a post-Twombly and Iqbal world sufficient data exist to begin to test various normative claims regarding whether the two procedure decisions amplify the potential impact of judge ideology. And in a new paper, Politics, Identity, and Pleadings Decisions on the U.S. Courts of Appeals, Stephen Burbank (Penn) and Sean Farhang (Berkeley) set out to do just that.

To do so, the authors drew a random sample of 700 cases from the universe of cases post-Iqbal through 2019 that involved a Court of Appeals review of a district court decision on whether to dismiss for failure to state a claim under Rule 12(b)(6). The paper focuses on "the extent to which the party (of appointing president), gender, and racial composition of panels are associated with their disposition of 12(b)(6) appeals." Key findings include "that panels with women and non-white judges are substantially more likely to rule in favor of a plaintiff reaching discovery in other civil rights claims, an important and cross-cutting civil rights category amounting to a quarter of 12(b)(6) appeals in our data, but that race and gender are insignificant outside that substantive area. Party is insignificant across the board in the random sample."

Data on Prosecutorial Discretion By Gender

While a robust judicial outcomes research literature continues to mature and evolve, a critical pre-judicial outcome stage--the exercise of prosecutorial discretion--remains relatively understudied. Understanding the prosecutor's role is crucial as they "exercise unfettered decision-making authority throughout criminal cases by selecting whom to charge, which charges to bring, and what plea bargains to accept." While prosecturial discretion increasingly garners critical public attention, scholarly attention remains hamstrung by a paucity of helpful, systematic data.

In a recent paper, Gender Favoritism Among Criminal Prosecutors, Stephanie Holmes Didwania (Wisconsin) makes some intriguing headway by exploring individualized evidence about prosecutorial decision-making with the benefit of data from more than 130,000 federal criminal defendants sentenced from 2002 through 2016.

The paper's individualized line prosecutor-level data are, of course, key, and the data set is the product of two separate data sources: U.S. Sentencing Commission data (for the defendants) and the Executive Office for United States Attorneys (EOUSA) that regularly publishes case-level data from the 93 USAOs located throughout the United States. These data originate from the Legal Information Office Network System (LIONS), which is the computer program that the EOUSA uses to track and manage case loads. After employing various filters, what emerged includes data on "131,207 defendants sentenced in 63 of the 94 federal district courts in the United States, comprising 172 unique courthouses, 1,498 unique prosecutors, and 1,589 unique courthouse-prosecutor combinations."

Despite the granular data, the paper's empirical execution is not without important limitations. One limiting factor is that line prosecutors are not randomly assigned to cases. Another is that the defendant's underlying criminal conduct is not observable to the researcher. As this paper explores gender-based in-group favoritism, the non-random case assignment is important as "cases in which prosecutors and defendants match on gender are systematically different from those in which prosecutor and defendant mismatch."

Limitations aside, the paper's key findings include some intriguing gender-based differences. While the paper finds that male and female prosecutors exhibit small and statistically insignificant differences in their treatment of defendants overall, evidence of relative leniency towards defendants of their own gender exists. And for the defendants, such gender-based in-group favoritism is not inconsequential. "The leniency associated with gender-matching ultimately translates into significantly lower sentences: a defendant can expect to earn a roughly five-month shorter sentence when their prosecutor is their own gender compared to when their prosecutor is the opposite gender, which represents a roughly eight percent reduction in sentence length at the mean." The paper's abstract follows.

"Prosecutors enjoy wide discretion in the decisions they make but are largely unstudied by quantitative empirical scholars. This paper explores gender bias in prosecutorial decision-making. I find that male and female prosecutors exhibit small and statistically insignificant differences in their treatment of defendants overall but demonstrate relative leniency towards defendants of their own gender. Such favoritism at charging translates into a sentencing gap of roughly five months of incarceration for defendants who are paired with an own-gender prosecutor versus an opposite-gender prosecutor, which represents a roughly eight percent reduction in sentence length at the mean. The estimates do not appear to be driven by differences in case assignments for male and female prosecutors."

The Fourth Amendment Post-Carpenter

The long-anticipated collision between Fourth Amendment rights and technology has largely arrived. On the one hand, traditional legal doctrine implies that "information shared with a third party like a bank or telephone company was not protected by the Fourth Amendment." On the other hand, the volume of information now exposed to third parties has "exploded in the internet age, encompassing virtually every kind of digital data generated by internet and cell phone users." In 2018, the Court weighed in and its Carpenter decision was deemed "revolutionary" by holding, in part, that "individuals can retain Fourth Amendment rights in information they disclose to third parties." Specifically, the Court ruled that government agents had to "obtain a warrant before collecting cell phone location data that showed virtually everywhere a suspect had travelled over a seven-day period."

How lower courts understand and have operationalized the Carpenter decision is the focus of The Aftermath of Carpenter: An Empirical Study of Fourth Amendment Law, 2018-2021, by Matthew Tokson (Utah). The paper examines all 857 federal and state court decisions that cite to Carpenter from the decision's 2018 publication through March 2021. Among the salient factors driving lower court outcomes include: "the revealing nature of the data at issue, the amount of data collected, and the automatic nature of data disclosure, and clearly do not include the number of persons affected." An excerpted abstract follows.

"[The paper] examines disagreements among lower courts about the scope and breadth of Carpenter, as some courts apply its concepts expansively while others attempt to narrow it from below. It explores how state courts apply federal constitutional law, blending federal and state interests in unique ways. And it analyzes the enormous practical impact of the “good faith exception” to the exclusionary rule, which permits the government to use unconstitutionally obtained evidence to convict defendants if such evidence was collected in reliance on prior law. Based on these findings, the Article suggests alternative factors and paradigms that courts can adopt to more effectively address Fourth Amendment questions going forward. In addition to its many contributions to the Fourth Amendment literature, the Article is the most comprehensive study to date of the jurisprudential impact of a Supreme Court case in the years following its publication."

Pro Se Civil Litigation in Federal Courts

Public and scholarly attention to pro se litigation is growing. In Mapping the Civil Justice Gap in Federal Court, Roger Michalski (Oklahoma) and Andrew Hammond (Oklahoma) provide a helpful empirical perspective.

Careful attention to pro se litigation is warranted. The sheer volume of pro se litigants will surprise many, if not most. According to the authors' calculations, last year 12 percent of the non-prisoner civil cases filed in federal district courts (more than 25,000 cases) were pro se. Aside from the scope of this issue, "pro se litigants raise unique practical, doctrinal, and normative concerns for the legal academy, the bar, and the federal bench.”

Not surprisingly, methodological challenges limit empirical assessments of pro se litigation. While the paper is appropriately mindful of these limitations, it nonetheless does not "let the perfect be the enemy of the good." As such, the paper analytically confines itself to describing "where pro se litigants live and identify the communities they inhabit." To be sure, while a modest contribution it nonetheless  adds to the knowledge base. That said, analytic limitations endure. "None of our claims speak to the characteristics of pro se litigants directly, only indirectly. This is an important limitation of this study and raises bias concerns. Bias might creep in because it is possible that pro se litigants are systematically atypical of the neighborhoods in which they live."

To accomplish its goals, and limitations aside, the paper exploits "a massive data set of 2.5 million federal dockets from a ten-year period, we obtained addresses of non-prisoner pro se litigants. We then geolocated these addresses and cross-referenced that information with demographic and economic Census data." After various data exclusions (e.g., prisoner suits and cases involving habeas, immigration/deportation cases, naturalization, and bankruptcy claims), what remained included approximately 20,000 federal civil cases per year.

The papers dwells on two core findings. One involves pro se litigants' "profound ordinariness." A second finding involves exceptions to the first. "Even after accounting for population and income, pro se litigants are more likely to reside in communities that are not homogenously white. And numerous rural communities feature fewer pro se litigants than expected.” An excerpted abstract follows.

"Unrepresented litigants make up a sizable and normatively important chunk of civil litigation in the federal courts. Despite their importance, we still know little about who these pro se litigants are. Debates about pro se litigation take place without sufficient empirical information. To help fill some of the gaps in our understanding of pro se litigants, this Article takes a new approach by mapping where pro se litigants live."

How Lower Courts Digest Supreme Court Decisions: Data on the Fourth Amendment After Carpenter (2018)

Technical advances and their implications for civil liberties collided anew in the Supreme Court's 2018 Carpenter v. U.S. decision. In that case, the Court made clear that "individuals can retain Fourth Amendment rights in information they disclose to a third party." The Court specifically ruled that the government needed a warrant before collecting cell phone location data that revealed a suspect's location over a seven-day period. In today's Internet age, Carpenter has the potential to expand privacy protections over a wide and deep swath of sensitive digital information. Insofar as the decision's practical contours and boundaries are, unsurprisingly, vague, "scholars and lower courts have tried to guess at what the law of Fourth Amendment searches will be going forward." Equally unsurprising, these guesses about Carpenter's meaning diverge.

In a recent paper, The Aftermath of Carpenter: An Empirical Study of Fourth Amendment Law, 2018-2021, Matthew Tokson (Utah) subjects various predictions and claims about Carpenter's implications to data. In particular, the paper analyzes "all 857 federal and state judgments citing Carpenter from its publication in June 2018 through March 2021. Relying on this unique, hand-coded database, the Article illuminates both the present and future of Fourth Amendment law.” Among the paper's findings include that: "factors that drive lower court decisions clearly include the revealing nature of the data at issue, the amount of data collected, and the automatic nature of data disclosure, and clearly do not include the number of persons affected." An excerpted abstract follows.

"[The paper] identifies the factors that drive modern Fourth Amendment search decisions—and those that fail to drive them. It examines disagreements among lower courts about the scope and breadth of Carpenter, as some courts apply its concepts expansively while others attempt to narrow it from below. It assesses how state courts apply federal constitutional law, blending federal and state interests in unique ways. And it analyzes the enormous practical impact of the 'good faith exception' to the exclusionary rule, which permits the government to use unconstitutionally obtained evidence to convict defendants if such evidence was collected in reliance on prior law. Based on these findings, the Article explores alternative directions that courts may take as they continue to refine Fourth Amendment law and address novel surveillance technologies. In addition to its many contributions to the Fourth Amendment literature, the Article is the most comprehensive empirical study to date of the jurisprudential impact of a Supreme Court case in the years following its publication."

Copyright Injunctions After eBay and the "Hysteresis Hypothesis"

Interest in a recent paper, The Hysteresis Thesis: An Empirical Study of Copyright Injunctions After eBay, by Matthew Sag (Loyola-Chi) and Pamela Samuelson (Berkeley), should extend beyond traditional IP/copyright borders. Here's why.

First, the IP/copyright implications. Prior to the Supreme Court’s landmark 2006 eBay decision, courts routinely issued injunctions in both patent and copyright cases upon a finding of actual or likely infringement of plaintiffs’ rights. Even though most scholars expected that the eBay decision would have an immediate and transformative effect on the grant of permanent injunctions in patent infringement cases, Jiari Liu's 2012 paper reported that in the first four years after the eBay ruling, courts rarely cited that decision and continued to grant injunctions at a very high rate. Sag and Samuelson's paper, however, concludes that Liu's main finding (and conventional wisdom) were "always somewhat erroneous and are now clearly outdated."

Second, the implications of the "how" and "why" of Sag and Samuelson's paper spill-over into empirical legal studies more broadly. Sag and Samuelson began their study with an eye towards "the hypothesis of hysteresis or delayed effect: the simplest way to think about this theory is that, if it is correct, we should see a greater effect in the third and fourth periods than in the 2007-2010 period immediately following eBay."

To do so, they exploited a data set of "336 reported U.S. federal court copyright injunction cases decided between January 1, 2000 and December 31, 2019. We begin with the pre-eBay era, from 2000 to 2006. Our remaining three eras are 2007-2010 (the immediate aftermath era), 2011-2014 (the Salinger-Flexible era), and 2015-2019 (the consensus era)." What they find, in part, is that: "Our evidence shows that the citation rate for eBay has increased and that the corresponding grant rate for copyright injunctions has decreased."

They attribute their findings of an Hysteresis effect to "the doctrinal conservatism of lower courts and the parochial nature of appellate court precedents, it is unsurprising that there might have been some delay in the adoption of even the most obvious implications of new Supreme Court precedents such as eBay for injunction grants in copyright cases." An excerpted abstract follows.

"... This Article tests these claims and finds statistically significant differences in the grant rate for both preliminary and permanent injunctions in copyright cases after the eBay decision. The impression of eBay lacking effect appears to be a product of hysteresis—a time lag between cause and effect, as lower courts initially delayed but eventually embraced eBay. This study reveals, moreover, that the earlier empirical study was flawed by biased sampling, most notably by including a large number of unreported cases and default judgments. This Article offers a corrective analysis and provides a more accurate understanding of the copyright injunction landscape, both pre- and post-eBay. It shows that eBay was in fact a landmark case for copyright law, both in terms of jurisprudence—it is highly cited—and real-world effect— it has a dramatic impact on case outcomes."

Pretrial Detention and Release Experiments

As I have an admitted soft-spot for (even quasi-) randomized control trials as a research design, a recent paper by Javier Osorio (Arizona-Govt. & Pub Pol’y) et al., Reducing Pretrial Detention: A Randomized Intervention with Public Defenders in El Salvador, caught my eye. In it, the authors exploit a "randomized intervention in El Salvador designed to improve public defenders’ capacities to secure pretrial release for defendants.”

To note that pretrial detention is a problem is to note merely the obvious. And in countries facing acute security challenges, such as El Salvador, there exists enormous pressure on criminal justice authorities to limit pretrial release. The authors contend that the overreliance on pretrial detention is driven in part by lack of information about defendants’ flight risks and the prospect of their committing further crimes while awaiting trial. And this information deficit is “the consequence of deficiencies in the public defender system, upon which economically disadvantaged citizens rely to ensure legal representation.”

To assess their hypothesis, the authors exploit an unusual (by U.S. standards, anyway) experiment involving a randomly assigned treatment to public defenders in four municipalities in El Salvador. “The treatment included specialized legal training, an improved interview protocol, and enhanced communication channels with the defendants, their relatives, and other institutions. These components sought to improve public defenders’ ability to gather evidence that would support pretrial release.”

The authors found that the treatment “increased public defenders’ requests for pretrial release by nearly 10%. The intervention also helped public defenders to secure pretrial release of their clients by 4.4%, although these results are weaker and sensitive to model specification.” The paper's abstract follows.

"Approximately one third of the global prison population is in pretrial detention, waiting for trial. Overreliance on pretrial detention exposes defendants to harsh conditions, exacerbates jail overcrowding, increases recidivism, and favors criminal governance. What policies can resource strapped countries implement to effectively address excessive pretrial detention? Based on a theoretical model focused on institutional-level efforts, we evaluate an experimental intervention implemented in El Salvador intended to increase pretrial release requests and reduce pretrial detention. The intervention randomly assigned public defenders to receive specialized legal training, an improved interview protocol, material resources, and increased communication channels. We find that this inexpensive, scalable program increased pretrial release requests from public defenders by nearly 10% (0.228 standard deviations) and increased the success in securing pretrial release by 4.4% (0.114 standard deviations). Heterogeneous treatment effect analyses suggest that the program increased strategic litigation among the most experienced public defenders and has distinct effects for those accused of minor and severe crimes. We find no evidence that the mechanism explaining our results involves changes in public defenders’ attitudes or perceptions about their work environments. Criminal justice programs focusing on pretrial detention may help reduce prison overcrowding in high crime countries."

Data Resources Courtesy of the APSA Law & Courts Section

Our good friends and political science colleagues over at the APSA's Law & Courts Section recently updated the Section's website (click here). What is especially helpful is a sub-page (here) that provides links to the leading court data sets (SCOTUS, CA, district courts, as well as state supreme courts) along with related data assets (e.g., Martin-Quinn Scores, Judicial Common Space Scores).

Bridge Ideal Points--2021 Update

Michael Bailey (Georgetown--Govt.) has updated the Bridge Ideal Points estimates (click here) to cover years 1950 through 2020. These Ideal point estimates include the Supreme Court, presidential, and congressional ideology estimates and are designed to facilitate inter-institutional and cross-temporal comparisons.

While the Bridge Ideal Points compete somewhat with the Martin-Quinn scores in the literature, the two sets of ideology estimates correlate reasonably highly with one another. Despite the overlap, these two ideology estimates differ in two important ways.First, "Bridge Ideal Points are designed to be intertemporally comparable. Martin and Quinn scores assume the ideological cutpoints of Supreme Court cases are drawn from a common distribution across years, which makes them less comparable across time when the case load changes." Second, while Martin and Quinn scores only cover the Supreme Court justices, Bridge Ideal Points for justices are comparable to Bridge Ideal Points for presidents and members of Congress as well.

Regardless of one's preference, efforts to estimate justice ideology remain a staple in the judicial decisionmaking literature.

The Empirics of Rule 23(f)

While civil procedure scholars have spilled a lot of ink on class actions, until quite recently little of it is informed by data. To help remedy this, in a recent paper, Class Certification in the U.S. Courts of Appeals: A Longitudinal Study, Stephen Burbank (Penn) and Sean Farhang (Berkeley) exploit a class certification decision data set that includes USCA precedential panel decisions (1966 through 2017) and nonprecedential panel decisions (2002 through 2017). The data permit the authors to test various "conventional wisdoms," including those related to class certification appeal outcomes and Rule 23(f)'s potential influence. Among their findings includes, e.g., that, "contrary to conventional expectations, in the period since Wal-Mart and Comcast, plaintiffs have been winning certification appeals more frequently than they were formerly, and Rule 23(f) contributed to this recent success." More specifically, the paper notes "significant variation over time in appeal outcomes under Rule 23(f), with defendants far more successful than plaintiffs prior to Wal-Mart and Comcast, and relative parity after." An excerpted abstract follows.

“There is a vast literature on the modern class action, but little of it is informed by systematic empirical data. Mindful both that there have been few Supreme Court class certification decisions and that they may not provide an accurate picture of class action jurisprudence, let alone class action activity, over time, we created a comprehensive data set of class certification decisions in the United States Courts of Appeals consisting of all precedential panel decisions addressing whether a class should be certified from 1966 through 2017, and of nonprecedential panel decisions from 2002 through 2017.

…

We find significant variation over time in appeal outcomes under Rule 23(f), with defendants far more successful than plaintiffs prior to Wal-Mart and Comcast, and relative parity after. This variation suggests the hazards of generalizing about operation of that rule from experience in any particular period. Our models also show that, for reasons about which we can only speculate, interlocutory appeals since around 2000 have elicited more ideological voting behavior by judges, leading to greater polarization.”

Life Sentences "With" or "Without"

While significant public and scholarly attention rightly dwells on capital punishment, comparatively under-explored is a steady surge of life sentences without the possibility of parole ("LWOP"). While the Furman Court's "death is different" doctrine retains obvious salience and moral force, it should not obscure LWOPs from much-needed empirical inquiry.

To this end, in a recent paper, Life 'With' or 'Without'?: An Empirical Study of Homicide Sentencing, forthcoming in JELS, Michael O'Hear (Marquette) and Darren Wheelock (Marquette--Dept. Social and Cultural Sciences) do just that. While the paper's data set is limited to Wisconsin homicides, the authors do a deep data dive into 450 individuals convicted of first-degree intentional homicide between the start of 2001 and about mid-2018. The core findings are mixed. On the one hand, "intentional-homicide defendants who have more serious criminal histories are more likely to receive LWOP." On the other hand, "sentencing outcomes are influenced by the identity of the judge and prosecutor." The paper's abstract follows.

"The number of Americans serving sentences of life without the possibility of parole (“LWOP”) has grown rapidly over the past generation and now exceeds 50,000. Yet, little empirical research has been conducted on the determinants of LWOP sentences. The dearth of research on LWOP sentencing stands in sharp contrast to the many dozens of studies that have been conducted on the determinants of death sentences—studies that have consistently found that race, gender, and other questionable factors may influence sentencing outcomes. The present study is the first to employ a similar methodology to identify both case- and county-level variables that are correlated with the imposition of discretionary LWOP sentences.

More specifically, we have assessed the relationship between fifty different variables and LWOP decisions in 450 homicide cases in Wisconsin between 2001 and 2018. In our final model, we find seven variables that are correlated with sentencing outcomes. Of particular note, we find that judge and prosecutor personal characteristics are statistically significant correlates of LWOP decisions. We also find a significantly greater likelihood that LWOP sentences will be imposed in counties that are more Republican. We conclude with a proposal for a new LWOP sentencing process that may help to ensure that this very severe sentence is reserved for the most serious crimes committed by the most dangerous defendants."