Empirical Legal Studies

Plotting Kernal Density Estimates as Colored Areas

By default, kernel density estimates are plotted as lines, meaning curves:

According to recent StataList discussions (see here and here), however, plotting kernal density estimates as "areas" has become the standard in some fields. While for publication purposes the coding requires color, if such a possibility exists the resultant information conveyed is quite helpful as the following example illustrates.

SCOTUS Nominations, By the Nimbers and Over Time

Charles Cameron (Princeton--politics ) & Jonathan Kastellec (Princeton--politics) recently published a book, Making the Supreme Court (OUP), that tracks 90 years of POTUS appointments to the Supreme Court. The Book's description (below) makes clear what the authors attempt to achieve with their book. The authors' execution benefits from data, including on:

• Party and presidential interest in the court, including the use of “litmus tests” for Supreme Court nominees;
• The type of selection processes employed by the president;
• The background characteristics of Supreme Court nominees, and those who made the president’s “short list”;
• Interest group participation in Supreme Court confirmation battles;
• Media coverage of Supreme Court nominees;
• Public opinion of nominees;
• Senate roll call voting on Supreme Court nominees; and
• Simulations on the future ideological trajectory of the Supreme Court.

But what particularly caught my eye was that the authors make available the data (and related coding) that they used in their analyses (click here).

“Appointments to the United States Supreme Court are now central events in American political life. Every vacancy unleashes a bitter struggle between Republicans and Democrats over nominees; and once the seat is filled, new justices typically vote in predictable ways. Making the Supreme Court examines 90 years of American political history to show how the growth of federal judicial power from the 1930s onward inspired presidents, the political parties, and interest groups to shape judicial policy through appointments. The result is a new politics aimed squarely at selecting and placing judicial ideologues on the Court, which has transformed the Court into an ideologically driven and polarized branch. Based on rich data and qualitative evidence, Making the Supreme Court provides a sharp lens on the social and political transformations that created a new American politics.”

Levering AI in the Service of Contract Interpretation

Contract interpretation canons and theories continue to evolve. The latest foray seeks to lever AI.

In a recent article, Generative Interpretation, the authors, David Hoffman (Penn) and Yonathan Arbel (Alabama), advance a new approach designed to assist in determining contracting parties’ meaning. The authors' impulse behind generative interpretation, is, according to them, "simple:" they apply large language models (LLMs) to contractual texts and extrinsic evidence to predict what the parties would have said at contracting about what they meant. The paper's abstract follows.

“We introduce generative interpretation, a new approach to estimating contractual meaning using large language models. As AI triumphalism is the order of the day, we proceed by way of grounded case studies, each illustrating the capabilities of these novel tools in distinct ways. Taking well-known contracts opinions, and sourcing the actual agreements that they adjudicated, we show that AI models can help factfinders ascertain ordinary meaning in context, quantify ambiguity, and fill gaps in parties’ agreements. We also illustrate how models can calculate the probative value of individual pieces of extrinsic evidence.

After offering best practices for the use of these models given their limitations, we consider their implications for judicial practice and contract theory. Using LLMs permits courts to estimate what the parties intended cheaply and accurately, and as such generative interpretation unsettles the current interpretative stalemate. Their use responds to efficiency-minded textualists and justice-oriented contextualists, who argue about whether parties will prefer cost and certainty or accuracy and fairness. Parties—and courts—would prefer a middle path, in which adjudicators strive to predict what the contract really meant, admitting just enough context to approximate reality while avoiding unguided and biased assimilation of evidence. As generative interpretation offers this possibility, we argue it can become the new workhorse of contractual interpretation.”

"Anecdotes," Data, and Research Design

As I have neither an interest in nor inclination to engage substantively with an on-going scholarly "dust-up," I am content to let two papers that recently "crossed my desk" speak for themselves (compare Church & Williams with Baker & Bradt). That said, and independent of this exchange in particular and multidistrict litigation in general, a few points raised by Baker (Texas) and Bradt (Berkeley), in Anecdotes Versus Data in the Search for Truth about Multidistrict Litigation, have broader applicability.

One goal of empirical research, generally stated, is to generate results that are persuasively and adequately scaffolded by the underlying data and research design. Whether a "convenience sample" eliciting self-reports from an on-line survey can reflect an unbiased, random, and representative draw from an underlying population universe is not a given and requires further analyses. That is, for such a data-generating strategy the specter of selection bias lurks and must be addressed.

Moreover, even if one is comfortable that neither selection bias nor other data shortcomings pose methodological problems, additional research design issues persist. To be more specific, and as Baker and Bradt note, if a research question seeks to meaningfully engage with, e.g., plaintiffs' satisfaction with MDL litigation experience, data on respondents' (that is, plaintiffs in MDL litigation) satisfaction (or dissatisfaction) should be compared with an otherwise equally representative and unbiased sample of plaintiffs from non-MDL cases. Without an appropriate control group, the results lack a necessary reference point that facilitates interpreting the results.

Lower Fedreal Courts and Religious Claims: An Update

The Supreme Court's ongoing readjustment of long-standing Free Exercise and Establishment clause tensions has renewed scholarly attention to courts' resolutions of religious claims. And in a recent paper, Trump's Lower-Court Judges and Religion: An Initial Appraisal, Stephen Choi (NYU) et al. contribute to a growing empirical literature by levering data on Trump-appointed Circuit Court judges in Free Exercise Clause cases and comparing Trump-appointed judge outcomes with those of Non-Trump Republican- and Democratic-appointed judges.

What the authors find includes that "Trump’s judicial appointments are more religious but no less qualified than other judges," and that "Trump judges are more likely to vote in favor of the plaintiff than not only Democratic judges, but also judges appointed by other Republican presidents."

Notably, however, while Trump-appointed judges vote “pro-religion” (that is, in favor of a religious plaintiff asserting a free exercise claim) more frequently than other judges, this tilt distributes unevenly across various religions. For example, and as the table below illustrates, the authors find “... different voting patterns for Islam. For cases involving Islam, Trump judges vote in favor of Muslim plaintiffs in only 19.2% of the cases compared with 34.5% for Non-Trump Republicans and 48.0% for Democrats.” The authors note that the overall pattern that emerges is “consistent with conventional wisdom: Democrats tend to protect minority religions, and Republicans tend to protect Christianity (and possibly Judaism).”

 

 

The paper's abstract follows.

“It is widely believed that President Donald Trump’s judicial appointments reflected a strategy of appeasing evangelical Christians and other religious groups that favored a more conservative, Christian judiciary, and that in pursuing this strategy Trump sacrificed quality. We explore evidence for this theory by examining the biographies and credentials of Trump’s lower court nominees and the voting records of his judicial appointments to the circuit courts in free exercise cases. We find that Trump’s appointments to the lower courts have stronger or more numerous religious affiliations, and are affiliated with the Federalist Society and the National Rifle Association at unusually high rates, but are no less well credentialed than other judges are. We also find that Trump’s appointments to the circuit court more frequently vote in favor of Christian plaintiffs, and less frequently vote in favor of Muslim plaintiffs, in free exercise cases than judges appointed by other Republican presidents and by Democratic presidents.”

Fixed Effects vs. Random Effects

While debates persist, how fixed- and random-effect estimators differ and what they seek to accomplish, however, remain somewhat muddled in the various empirical literatures. A recent discussion on StataList, however, provides a rare moment of clarity (here) by nudging us to think about fixed- and-random effects as involving different tradeoffs (or preferences) between consistency and efficiency. As one commentator helpfully notes:

"The -fe- estimator is always consistent but inefficient if -re- is the way to go.
 The -re- estimator is inconsistent if -fe- is the way to go."

Skewed Data, Histograms & Axis "Breaks"

While skewed data distributions (e.g., a large proportion of zeros) invite an array of well-known methodological complications, they also complicated data presentation needs. For example, a standard histogram for a distribution that includes approx. 85% zeros looks something like the following:

Setting aside the methodological complications imposed by such skewed data, as it narrowly relates to descriptive data presentation needs one "dissenting" approach is to insert axis "breaks" (discussion here) that generates the following histogram:

Such a strategy may provide the optimal tradeoff between the visualization of "extreme" data and compression of the non-extreme data. This approach also has the virtue of keeping the underlying data in their raw rather than a transformed scale.

CELS 2023--Submission Deadline Extended

2023 Conference on Empirical Legal Studies

October 13-14, 2023

Call for Papers -- Submission Deadline Extension

University of Chicago Law School

The deadline for submitting papers to the 2023 Conference on Empirical Legal Studies (CELS) has been extended. The new deadline is Sunday, July 9, 2023 (11:59 PM Central Daylight Time).

Please note that this is the final deadline. The Conference website will not accept submissions after 11:59 PM (CDT) on Sunday, July 9, 2023.

To submit a paper, please visit the CELS 2023 website. We encourage preregistration of all experiments, and we encourage all authors to make their data, scripts, and code publicly available upon publication to the extent your methodology permits. We welcome replication studies and papers that develop and demonstrate empirical methods.

About: The University of Chicago Law School and the Society for Empirical Legal Studies (SELS) are pleased to host the 17th Annual Conference on Empirical Legal Studies (CELS) on October 13–14, 2023, in person at the University of Chicago. CELS is an interdisciplinary gathering that draws researchers from across the United States and the world, bringing together scholars in law, economics, political science, psychology, and other fields who are interested in the empirical analysis of law and legal institutions. Visit the conference website.

Questions about the submissions process should be directed to the conference organizers at: [email protected].

Data on Local Prosecutor Campaign Contributions & Electoral Outcomes

Building on prior descriptive work (here) that focuses on local prosecutor elections in the U.S., in a recent paper, Donating to the District Attorney, Carissa Hessick (UNC) et al., broaden their focus to assess how district attorney campaign contributions map onto electoral success. To inform the authors' interest in "the local politics of mass incarceration," the paper exploits a unique data set that links contribution and election results data. Merging these two data sets allows one to know whether a given recipient [local prosecutor candidate] was an "incumbent or challenger, the recipient’s vote share in both the primary and general election, and whether the election was contested in the primary election, the general election, or either election."

Such data provide a "more nuanced picture of prosecutorial politics, allowing us to make important distinctions between offices and interest groups, suggest new accountability measures, and better evaluate how elections can lead to reform." What the authors find propels them to conclude that "accountability mechanisms beyond elections are necessary to ensure prosecutors do not exercise their significant discretionary power unchecked." The paper's excerpted abstract follows.

“… The study offers a more complete empirical account of prosecutor accountability by analyzing contributions to local candidates as well as their election results. It details the amount of money in local prosecutor elections, including from interest groups, and the relationship between candidate fundraising and success. The stark differences across the country underscore that the more than two thousand local prosecutors are not a monolith; some offices are best understood as political, with contested elections and significant amounts of campaigning, while most appear more bureaucratic...”

Judicial Decisionmaking: Moving Beyond Case Outcomes

While conceding that "judges matter" as a general matter, a recent paper, Do Judges Matter?, considers a slightly narrower question: what about judges matters when it comes to the more mundane aspects incident to the business of judging?" More specifically, the authors, Yun-chien Chang (Cornell) and Geoffrey Miller (NYU), explore potential relationships "between a judge’s personal characteristics and the day-to-day business of courts – the writing of opinions, the use of citations, the frequency of dissent, and like factors."

Exploiting a hand-collected data set of 4,591 substantive decisions issued by 48 state supreme courts in 2003 (excluding Alaska and Hawaii), the paper finds that while state supreme courts themselves vary in how they carry out their judicial business (e.g., citation distributions, frequency of dissents), personal characteristics of judges, by contrast, have only a "negligible impact." The abstract follows.

"An extensive literature examines whether characteristics of judges correlate with votes on cases. These studies generally consider the judges’ votes on the merits of cases. Examining a data set of 4,591 decisions issued by 48 state supreme courts in 2003, we consider whether judges’ personal features affect their opinion writing. We find virtually no significant differences along any of the dimensions under review. Judicial characteristics matter only indirectly — ideological differences increase dissensus in public-law cases, which increases opinion length and citation numbers. Our study suggests that when carrying out the ordinary business of courts, judges are almost all the same."

Data on the Fourth Amendment's "Good Faaith" Exception's Application

While the Fourth Amendment is designed to protect criminal defendants against prosecutors' use of unlawfully obtained evidence, an important exception exists for evidence obtained by law enforcement officers who rely in good faith on existing legal authority. Thus, if police rely on a statute that turns out to be unconstitutional or invalid warrants or precedent, the evidence obtained is nonetheless admissible under the good faith exception.

Empirical analyses on how the good faith exception works "on the ground," however, are scant. To this end, a recent paper, The Reality of the Good Faith Exception, by Matthew Tokson (Utah) and Michael Gentithes (Akron), helps to fill a void.

Their paper draws from available published (and unpublished) judicial decisions (N=5,531) that mention the Fourth Amendment or "motion to suppress" across three separate months (July 2015, 2018, and 2021). Among the paper's findings includes a discernible uptick in the exceptions salience over time (see Figure 1, below).

Figure 1: Good Faith Exception (GFE) Prevalence Over Time - Proportion of Fourth Amendment Suppression Cases Involving the Good Faith Exception

The authors are correctly mindful that selection effects certainly influence their descriptive findings. The authors argue that selection effects in this context likely lead, on balance, to an under-counting of suppression hearings to the extent that "defendants and their attorneys likely decline to file suppression motions because of the good faith exception." (emphasis added)

Overall, the paper argues that "the [good faith] exception, originally confined to narrow circumstances, is turning into a protection for nearly any investigatory activity with an arguable connection to an existing precedent or statute" and that "[c]ourts rely on the exception with remarkable frequency, and a large proportion of courts invoking the exception avoid any substantive constitutional ruling, stunting the development of Fourth Amendment doctrine."

"Heatplots" for Correlation Coefficients Graphs

While I've never used one myself, I detect an upward trend in "heatplots" (example below) for basic descriptive correlation coefficient graphs.

A helpful discussion (along with coding suggestions) on what is possible with Stata's heatplot command is found here.

CELS 2023 -- Call for Papers

2023 Conference on Empirical Legal Studies

October 13-14, 2023

Call for Papers

University of Chicago Law School

The 2023 Conference on Empirical Legal Studies (CELS) is now accepting submissions. CELS will consider empirical papers spanning all areas of empirical legal studies. Authors are encouraged to submit works-in-progress; however, submissions should be completed drafts that include principal results. Submitted papers must be unpublished (and expected to be unpublished at the time of the conference).

The Paper Submission Deadline is Thursday, June 29, 2023 (11:59 PM Central Daylight Time). There is no charge for submissions.

To submit a paper, please visit the CELS 2023 website We encourage preregistration of all experiments, and we encourage all authors to make their data, scripts, and code publicly available upon publication to the extent your methodology permits. We welcome replication studies and papers that develop and demonstrate empirical methods.

Papers are selected through a peer review process. If accepted, authors will have an opportunity to submit a revised draft prior to the conference for presentation and discussion. Discussion at the conference includes assigned commentators and audience questions. Accepted papers will be made available to all conference participants.

About: The University of Chicago Law School and the Society for Empirical Legal Studies (SELS) are pleased to host the 17th Annual Conference on Empirical Legal Studies (CELS) on October 13–14, 2023, in person at the University of Chicago. CELS is an interdisciplinary gathering that draws researchers from across the United States and the world, bringing together scholars in law, economics, political science, psychology, and other fields who are interested in the empirical analysis of law and legal institutions. To visit the conference website, click here.

If you have questions about the submissions process, please contact the conference organizers at [email protected].

Lay Understandings of Civil Settlements

When it comes to resolving formal civil disputes, popular imagination, partly fueled by the media and entertainment industry, is dominated by "courtroom drama" and jury trials. Empirical work over the decades, however, paints quite a different picture, and one dominated by the "disappearing trial." Thus, as Jennifer Robbennolt (Illinois) et al. make clear in Settlement Schemas: How Laypeople Understand Civil Settlement, "for all the attention paid to the real and imagined drama of the courtroom, these depictions overlook the reality of how many disputes are resolved: through any means but trial."

In the service of seeking a better understanding of how the public understands civil settlements, the authors surveyed a representative sample of more than 1,000 U.S. adults. Key findings include that while the public conveyed a widespread general understanding about civil settlements, evidence of misconceptions about the details, including the role of a jury and settlement scope, also emerged. The paper's abstract follows.

"What does the public think it means to ‘settle’ a civil case? Most legal disputes in the United States end in an agreement to settle, but little is known about what laypeople think about settlement. To fill this gap, we took a direct approach: we asked a nationally representative sample of U.S. adults – more than one thousand of them – basic questions about settlement. We found widespread understanding about the essential nature and frequency of settlement, but persistent, though not universal, misconceptions about the details, including the role of a jury and settlement scope. Because settlement is such a pervasive part of the U.S. legal system, the system’s legitimacy turns in part on how the public understands and views civil settlement. The survey reported here provides a foundational study of the understandings and framework – the schemas – that the public bring to settlement."

Importing Non-Stata Data Sets Into Stata

While perhaps old-hat for some (many), a quick new Stata tutorial video (click here) walks viewers through some of the basics incident to data importation. For old-time users (including those who may reflexively turn to StatTransfer programs for conversions), increasingly easy--and a wider array of--data transfer options remain much-welcome news.

Sentencing as a Product of Plea Agreements

When it comes to explaining criminal sentences, conventional wisdom emphasizes either judges or, given that criminal convictions largely flow from negotiated plea agreements, prosecutors. A closer look, however, reveals far more nuance and complexity and a dynamic process involving multiple parties concurrently rather than a static event dominated by one actor.

In a recent "closer look," Jeffrey Bellin (William & Mary) & Jenia Turner (SMU) explore 77 plea bargains in the first half of 2019. Noting that comprehensive empirical work on judicial responses to negotiated plea deals is sparse, In Sentencing in an Era of Plea Bargains makes a few helpful contributions. First, the authors' deep dive into 77 pleas bargains, though descriptive, revels some initial, general contours of these agreements. Notably, while judges ultimately approved all of the 77 plea bargains in the paper's sample, judges departed from the negotiated sentencing recommendation in about one-third of the cases. Also notable is that in virtually all instances where the judge departed from the negotiated sentence, she did so in a direction that favored greater leniency. Second, in separate analyses the authors examine 31 plea agreements that elicited judicial rejections. Somewhat perplexingly, judges rejected most plea agreements (74%) because they felt the recommended sentence was too lenient.

Obviously, descriptive studies that lack statistical power are limited in important ways. At the same time, however, such studies can help by pointing future researchers in potential helpful directions. The paper's (excerpted) abstract follows.

“The literature offers inconsistent answers to a question that is foundational to criminal law: Who imposes sentences? Traditional narratives place sentencing responsibility in the hands of the judge. Yet, in a country where 95 percent of criminal convictions come from guilty pleas (not trials), modern American scholars center prosecutors – who control plea terms – as the decider of punishment. This Article highlights and seeks to resolve the tension between these conflicting narratives by charting the pathways by which sentences are determined in a system dominated by plea bargains.

After reviewing the empirical literature on sentence variation, state and federal plea-bargaining rules and doctrines, and conducting some empirical analysis of our own, we conclude that neither of the competing narratives is correct. Sentencing in the United States has become a dynamic process with substantial contributions from multiple actors, not a static event controlled by any single actor.”

How Summary Judgment Changes Vary Across Subject Areas

The Supreme Court upended summary judgment doctrine with a trilogy of cases in 1986. At the time--and largely persisting into current prevailing wisdom--it was assumed that the 1986 changes made summary judgment comparatively more available and, thus, of greater benefit to defendants. While empirical evidence remains somewhat scant, emerging findings paint a far more complex and nuanced picture.

A recent contribution to this research literature comes from Jonathan Nash (Emory) and Daniel Sokol (USC). In The Summary Judgment Revolution that Wasn't, the authors lever six separate (though related) datasets that draw from federal court decisions in three discrete areas (antitrust, civil rights, and securities regulation) and exploit the 1986 trilogy with pre- and post-intervention comparisons. At bottom, the paper finds "no evidence of the trilogy influencing summary judgment decision making in securities regulation, and at best mixed evidence of such an effect in civil rights cases. In contrast, we find that the trilogy did have a strong statistically significant effect on decision making in antitrust cases, at both the district court and court of appeals levels." The abstract follows.

"The U.S. Supreme Court decided a trilogy of cases on summary judgment in 1986. Questions remain as to how much effect these cases have had on judicial decision making in terms of wins and losses for plaintiffs. Shifts in wins and losses and what cases get to decisions on the merits impact access to justice. We assemble novel datasets to examine this question empirically in three areas of law that are more likely to respond to shifts in the standard for summary judgment: antitrust, securities regulation, and civil rights. We find that the Supreme Court’s decisions had a statistically significant effect in antitrust, an ambiguous effect in civil rights cases, and no effect in securities regulation. We also find that, in the trilogy’s wake, antitrust appellate cases were far more likely to cite trilogy cases—and in particular the one trilogy case that was an antitrust case—than appellate cases in the other areas. This suggests that the lone trilogy case that arose in antitrust had an effect on decision making in that field, but that the trilogy had a limited effect across other substantive areas. This finding differs from Twombly and Iqbal where an antitrust decision ultimately reshaped the entire body of law around motions to dismiss."

Experimental Judicial Decisionmaking Studies & External Validity

For experimental judicial decisionmaking studies, the "gold-standard" research design includes using actual judges as participants. My Cornell colleague (and good friend) Jeff Rachlinski is a leading scholar in this space. For an array of reasons (including practical reasons), too much of the experimental research in this field is hampered by researchers' use of students (including law students) as the experimental participants. To the extant that researchers seek to generalize results from judicial decisionmaking studies to the "universe of judges," conventional  wisdom suggests that the use of students in these experiments raises important external validity questions.

Of course, such (and most) "conventional wisdoms" rest on empirical assumptions. And in a recent paper, Can Law Students Replace Judges in Experiments of Judicial-Making?, Holger Spamann (Harvard) and Lars Klohn (Humboldt) submit this question to an experiment. The answer to the question they pose in their paper's title?: "unfortunately, no." The paper's abstract follows.

"Experimental research on judicial decision-making is hampered by the difficulty of recruiting judges as experimental participants. Can students be used in judges’ stead? Unfortunately, no. We ran the same high-context 2×2 factorial experiment of judicial decision-making focused on legal reasoning with 31 U.S. federal judges and 91 elite U.S. law students. We obtained diametrically opposed results. Judges’ decisions were strongly associated with one factor (sympathy, i.e., bias) but not the other (law). For students, it was the other way around. Equality between the two groups is strongly rejected. Equality of document-view patterns—a proxy for thought processes—and written reasons is also strongly rejected."

A Quick Refresher on T- vs. F-Tests

While I haven't thought about this since grad school, a recent discussion on the StataList (here) provides a helpful, quick refresher on when F- and T-Tests converge and when they do not (and why).

Specific Performance, Expectations, and Elon Musk

For one brief moment in time in 2022, to some it looked as though a Delaware judge was poised to order Elon Musk to purchase Twitter, over his objections, due to a contract breach. Had that happened, as Tess Wilkinson-Ryan (Penn) et al. note in Expecting Specific Performance, it would have been "the highest-profile specific performance order in living memory."

The incident helped prompt Wilkinson-Ryan and colleagues to explore intuitions on specific performance through a series of mixed-method experiments. And what they find, on balance, are "fragile" intuitions. Levering a mix of survey and experimental data, the authors find that "ordinary people think that courts will give them exactly what they bargained for after breach of contract; in other words, specific performance is the expected contractual remedy." They go on to argue that "the default expectation of equitable relief is a widespread but malleable intuition—and that even a fragile legal intuition has practical consequences, individually and systemically." Indeed, updating subjects with the prospect of money damages for a contract breach loosens initial default expectations for specific performance. The paper's abstract follows.

“Using a series of surveys and experiments, we find that ordinary people think that courts will give them exactly what they bargained for after breach of contract; in other words, specific performance is the expected contractual remedy. This expectation is widespread even for the diverse array of deals where the legal remedy is traditionally limited to money damages. But for a significant fraction of people, the focus on equity seems to be a naïve belief that is open to updating. In the studies reported here, individuals were less likely to anticipate specific performance when they were briefly introduced to the possibility that courts sometimes award damages in contract disputes.

We argue that the default expectation of equitable relief is a widespread but malleable intuition—and that even a fragile legal intuition has practical consequences, individually and systemically. In a follow-up experiment, we show that subjects are more interested in the prospect of efficient breach when they know that money damages are a possible remedy. This finding suggests that the mismatch between what people assume the law will do (specific performance) and what it actually does (money damages) sometimes encourages performance. We consider the potential for exploitation of this tendency. Finally, we offer some suggestions about how law and psychologists should elicit folk beliefs about legal rules and remedies."

2023 Northwestern Main and Advanced Causal Inference Workshops

Co-organized by Bernie Black (Northwestern) and Scott Cunningham (Baylor—econ), the 12th annual workshop on Research Design for Causal Inference will be held at Northwestern Law School in Chicago, IL.

Main Workshop:  Monday – Friday, August 7-11, 2023; Advanced Workshop:  Monday – Thursday (midday), August 14-17, 2023; and Optional Machine Learning Primer:  Sunday afternoon, August 13, 2023.

Main Workshop Topics/Faculty: Introduction to Modern Methods for Causal Inference:  Donald B. Rubin (Harvard Univ.), Matching and Reweighting Designs and Difference-in-Differences: Jens Hainmueller (Stanford Univ.), Regression Discontinuity:  Matias Cattaneo (Princeton Univ.), and Instrumental variables:  Rocio Titiunik (Princeton Univ.).

Advanced Workshop Topics/Faculty: Applications of machine learning to causal inference:  Christian Hansen (Univ. Chicago), Advanced matching and balancing methods: Jeffrey Wooldridge (Michigan State Univ.), Advanced panel data methods:  Yiqing Xu (Stanford Univ.), and Empirical Bayes Methods:  Christopher Walters (UC Berkeley).

In person-registration is limited to 125 participants for each workshop. There will also be a Zoom option, but please come in person if you can.  The online experience is not the same.

For further information and registration: (click here)

ChatGPT & Statistical Analyses

I suspected discussions about this were inevitable; I wasn't disappointed. Where this may land over time in terms of coding, data, etc. is anyone's guess.

CELS 2023--Save the Date

2023 Conference on Empirical Legal Studies

Save the Date

October 13-14, 2023

The University of Chicago Law School

David Rubenstein Forum
1201 E 60th St
Chicago, IL 60637

19th Asian Law and Economics Association (AsLEA) Annual Meeting -- Call for Papers

The conference's central theme is "Rationality, Norms, and the Law." The in-person conference, scheduled for 8-9 August 2023, at the Academia Sinica, Taipei, Taiwan, invites papers regarding the economic analysis of laws and institutions or the relationship between rationality and norms in the legal system.

Professor Kaushik Basu (Department of Economics and SC Johnson Graduate School of Management, Cornell University) and Professor Todd Zywicki (Antonin Scalia School of Law, George Mason University) are the keynote speakers of the 2023 AsLEA Conference.

Please submit the manuscripts or abstracts of your article or a session proposal, with a short bio in PDF or Word to [email protected] by May 31, 2023. We will make early decisions on a rolling basis and inform applicants of the results as soon as possible. There are no submission or registration fees for the 2023 AsLEA Conference. For hotel information click here.

Please visit the Asian Law and Economics Association website (click here) for updated information. If you have any questions, please send an e-mail to [email protected] or contact the local organizer. Note that Participants should take care of booking hotels and arranging travels by themselves. Neither AsLEA nor the local organizer is able to provide stipends or reimburse expenses of participants.

Local Organizer: Tze-Shiou Chien (Deputy Director & Research Professor, Institutum Iurisprudentiae, Academia Sinica; Vice President, AsLEA. [email protected]).

Criminal Records & Accuracy

Given the outsized implications for individuals, to note that criminal record keeping requires accuracy is to note merely the obvious. This is especially true in the employment context where "94% of employers conduct background checks" on job applicants. Growing employer demand for background checks has, in turn, fueled a dramatic increase in private sector service providers in this space. While access to and uses of criminal records have expanded, concerns persist about "serious data quality problems" in various criminal justice databases.

To explore this issue, in a recent paper, The Problem With Criminal Records, Sarah Lageson (Rutgers--crim. just.) & Robert Stewart (Maryland--crim. just.) compare criminal record reports for 101 New Jersey participants, all of whom had at least one criminal change in their history, across three sources: (1) the official state record; (2) a consumer reporting agency; and (3) an open source, public records data aggregator. The researchers then asked each of the 101 participants to review and audit the information retrieved from the three sources.

As the figure above visually implies, what the researchers found includes errors, and lots of them. Among the 2,406 individual charges reported by the three separate sources, only 8.3% of the participants' criminal charges (200/2,406) were accurately reported by all three sources. Moreover, 60% and 50% of participants had at least one false positive error on their regulated and unregulated background checks, and nearly all (90% and 92% of participants, respectively) had at least one false negative error. As this study is limited to individuals with criminal histories, it remains unclear whether errors also implicate those with no criminal history. The paper's abstract follows.

"Criminal records are routinely used by employers and other institutional decision-makers, who rely on their presumed fidelity, to evaluate applicants. We analyze criminal records for a sample of 101 people, comparing official state reports, two sources of private sector background checks (one regulated and one unregulated), and qualitative interviews. Based on our analysis, private sector background checks are laden with false positive and false negative errors: 60% and 50% of participants had at least one false positive error on their regulated and unregulated background checks, and nearly all (90% and 92% of participants, respectively) had at least one false negative error. We define specific problems with private sector criminal records: mismatched data that create false negatives, missing case dispositions that create incomplete and misleading criminal records, and incorrect data that create false positives. The prevalence of error in private sector background checks limits access to social opportunities ranging from employment, education, and housing."

Call for Papers: 7th Annual Penn-NYU Empirical Contracts Workshop

As hints of our "pre-COVID" lives begin to emerge, including in-person conferences, I am delighted to pass along the following call for papers from a long-time ELS-blog reader:

The 7th Annual Penn-NYU Empirical Contracts Workshop is scheduled to place in-person at the New York University School of Law on 23 May 2023.

The Workshop is a forum for the presentation and discussion of early and mid-stage projects analyzing contract law and practice from a variety of empirical perspectives. Papers are selected through a peer review process.  Attendees of the conference are expected to cover their own travel and lodging costs, but there are no conference fees.

Submitted papers must be unpublished (and expected to be unpublished at the time of the conference). If accepted, authors will have an opportunity to submit a revised draft prior to the conference for presentation and discussion. Please note that accepted papers will be made available to all conference participants.

Please note that the Workshop paper submission deadline is 1 March 2023.

Organizing committee:  Dave Hoffman (Penn); Florencia Marotta-Wurgler (NYU) and Tess Wilkinson-Ryan (Penn).  Please email submissions to: [email protected]   

Modeling Rare Events--SCOTUS Clerkships

Of the approximate 36,000 law school students who graduate each year, only 36 or so end up securing a SCOTUS clerkship. While a handful of biographical and anecdotal accounts exist, very few systematic data exist on the "1-in-1,000" clerkship lottery.

A recent paper by Tracey George (Vanderbilt), Albert Yoon (Toronto), and Mitu Gulati (Virginia), Some Are More Equal Than Others: U.S. Supreme Court Clerkship, digs into 40 years worth of data and provides rich granularity about this statistical rare event. In the main, while some unexpected twists emerge, what the authors find generally comports with perceived wisdoms. For example, a disproportionate number of clerks emerge from five elite law schools. One interesting change, however, is that while a SCOTUS clerkship once led "primarily to government and universities [law faculties]," former clerks now increasingly land at "law firms and corporations." An excerpted abstract follows.

"... Yet, we lack a clear story about who wins this clerkship lottery. This Essay seeks to provide that story. Our analysis relies on new datasets of all Court clerks who served between 1980 and 2020, including the details of their path to the Court and their road after. We amend and expand on theories of success in this important labor market. All clerks are highly qualified, yet the same is true for many not chosen. We find that educational pedigree, as opposed to academic performance or any other qualification, often distinguishes the winners from the also-rans. The Court clerkship selection process, like other elite labor markets, proves to be a blend of status and merit where status often prevails. Our analysis does not end there, however. We go on to look at where this forty-year cohort is currently working and confirm that once attained, a Court clerkship does lead to a bounty of opportunities including a return to the Court as a justice. Thus, the Court clerkship lottery is an important labor market not only to lawyers but also to society writ large. In the elite legal labor market, some people are, in fact, more equal than others."

More Evidence of a Racialized Death Penalty

Notably, race does not emerge as a central concern in the Court’s Furman decision, now more than fifty years old. In the decades since Furman, however, empirical evidence continues to mount documenting that "racial discrimination infects the administration of the death penalty." A recent paper by Jeffrey Fagan et al. (Columbia), Getting to Death: Race and the Paths of Capital Cases after Furman, represents a recent and important contribution to this literature. Researchers visited each judicial circuit in Georgia to record the details of every murder case that entered the courts from 1994–2005. Analyses of the 1,317 death-eligible murders suggest that “race continues to matter, and that the statutory aggravators that distinguish death cases from other killings are themselves racialized.” A more detailed description of the paper's core findings appears in the Abstract (below).

“Decades of research on the administration of the death penalty have recognized the persistent arbitrariness in its implementation and the racial inequality in the selection of defendants and cases for capital punishment. This Article provides new insights into the combined effects of these two constitutional challenges. We show how these features of post-Furman capital punishment operate at each stage of adjudication, from charging death-eligible cases to plea negotiations to the selection of eligible cases for execution and ultimately to the execution itself, and how their effects combine to sustain the constitutional violations first identified 50 years ago in Furman. Analyzing a dataset of 2,328 first- degree murder convictions in Georgia from 1995–2004 that produced 1,317 death eligible cases, we show that two features of these cases combine to produce a small group of persons facing execution: victim race and gender, and a set of case-specific features that are often correlated with race. We also show that these features explain which cases progress from the initial stages of charging to a death sentence, and which are removed from death eligibility at each stage through plea negotiations. Consistent with decades of death penalty research, we also show the special focus of prosecution on cases where Black defendants murder white victims. The evidence in the Georgia records suggests a regime marred less by overbreadth in its statute than capriciousness and randomness in the decision to seek death and to seek it in a racially disparate manner. These two dimensions of capital case adjudication combine to sustain the twin failures that produce the fatal lottery that is the death penalty.”

Data on Whether Lawyers Add Value in IPOs

A common complaint is that lawyers provide the "grit" while the I-bankers provide the "grease" when it comes to commercial transactions, especially IPOs. Existing research on lawyers' transactional roles focuses on two alternative perspectives: The skilled advice and the passive execution hypotheses. To test these two competing hypotheses, in a recent paper, Do Lawyers Matter in Initial Public Offerings?, Bates (ASU--Finance) et al. draw from the Thomson Financial Securities Data Company (SDC) new issues database to identify U.S. IPOs over the period of 1986–2016. After filtering, the study's final sample includes data on 7,460 IPOs.

What the paper finds may--or may not--surprise. Specifically, the results imply that issuer law firm fixed effects account for an "economically significant portion of the variation in IPO outcomes, even after accounting for underwriter fixed effects." Specifically, the paper notes that "legal advisor fixed effects explain approximately 16.7% of the variation in IPO underpricing, which is about 70% of the variation attributable to underwriters documented in this and prior studies." Thus, if these effects are real, the paper evidences "the economic importance of legal advisors in the IPO process." The abstract follows.

"This study provides novel evidence that law firms representing issuers have a material impact on economic outcomes in the IPO market. Fixed effects for legal counsel explain a significant portion of the variation in IPO underpricing, comparable to underwriter fixed effects documented in prior studies. Issuer law firms also exhibit differential skill in moderating the likelihood of IPO-related litigation, and have an idiosyncratic influence on the structure of IPO disclosures. Accounting for endogenous selection, we document that law firms associated with lower rates of IPO-related litigation are also associated with less first-day underpricing and receive higher fees from their clients."

An Online Resource of Note

For those new to ELS, and even those with more sophisticated questions, UCLA's statistical consulting site, festooned with an array of resource links (here), remains quite helpful. While the site continues to evolve over time, some well-organized and useful links will help orient and guide those engaged in ELS-related work. Examples include a quick reference chart on "choosing the correct statistical test" (which includes command syntax for SAS, Stata, SPSS, & R), illustrated examples of how to interpret annotated output for standard commands, and robust "FAQ" links that are, again, program-specific (complemented by a "general FAQ" link (here)).

Judges' Financial Holdings, Case Outcomes, and Recusal

Fundamental to the notion of a fair trial and the rule of law is the principle that a judge shall apply the law impartially and free from the influence of any personal biases. When real or perceived conflicts of interest emerge, judges are expected to recuse themselves. Despite such an expectation, however, recent Wall Street Journal reporting revealed many cases where judges violated U.S. law and judicial ethics (28 U.S.C. Section 455) by overseeing cases that involve companies in which they or their family-owned stock. Unsurprisingly, several of the cases with conflicts have been appealed.

While Chief Justice John Roberts downplayed the significance of the Wall Street Journal reporting, characterizing the judges' failures to recuse as "isolated violations" and "unintentional oversights," empirical explorations of the potential influence of judges' financial holdings on case outcomes remain rare. To address this scholarly gap, a recent paper, Judge Financial Holdings and Case Outcomes: Evidence from Judge Financial Disclosures, examines whether there is a relation between the financial holdings of judges and the outcomes of the trials over which they preside.

To construct the judges' investment data set, Harit (Univ. Texas-Dallas—mgmt.) et al. extracted judge financial disclosure data from the Free Law Project website. Their sample includes 33,726 annual disclosure reports involving 3,250 federal judges from 2000 to 2020. The authors lever the random assignment of cases to judges and examine whether a judge's rulings are related to his or her stock ownership in one or more of the parties in the case. The paper finds robust evidence that judges rule more favorably and take longer when deciding over conflicted cases as compared to unconflicted ones. It also notes that investors are positively surprised by these outcomes. The abstract follows.

"This paper examines the association between a judge's financial holdings and the outcome of legal cases. Using novel data on financial disclosures of judges and the civil case information of public firms across district courts in the United States between 2000 and 2021, we document multiple instances in which judges fail to recuse themselves in cases where they hold common stock. We find robust evidence that judges rule more favorably and take longer when deciding over conflicted cases as compared to unconflicted ones. We also find that investors are positively surprised by these outcomes, as reflected by stock returns to trial outcomes. Collectively, our results provide the first large-scale evidence of the relation between judges' financial holdings and case outcomes."

Interpreting DID Output

While one may quibble about how the questioner designed her DID estimation, the question itself--how to understand and interpret standard DID output (in Stata)--elicited some helpful replies in a recent StataList discussion (here).

The Two-Part Metascience 2023 Confernece(s)

At the request of a loyal Elsblog reader--who is also helping organize the Metascience 2023 conference--I pass along the following.

Metascience is an organization that promotes "global gathering for knowledge sharing, community building, and opportunities to define a roadmap of research and intervention priorities to accelerate science." To this end, "Metascience 2023" involves two separate--though related--conference components. The in-person component is scheduled for 9-10 May 2023, at the National Academy of Sciences in Washington, DC (USA). The organizing group is also mounting a free virtual component, set for "late April and early May," 2023.

For anyone interested in additional information, click here for general conference information, and here for the conferences' call for papers/proposals.

Civilian Complaints Against Police & Prosecutor "Upcharging"

Citizens’ engagement with police serves as a valuable resource. One particular form of citizen engagement—civil complaints—can impose a serious cost to the targeted police officer as well as the police department. To try to reduce the potential negative effects of a citizen complaint against law enforcement agents, other criminal justice actors could act to re-frame the encounter such that the optics look unfavorably on the complaining citizen. Since a defendant cannot seek monetary damages in civil court unless his/her conviction or sentence was declared invalid, prosecutors may increase the number or the severity of charges that the defendant faces to propel the defendant toward accepting a plea deal, which would prevent the defendant from seeking further financial reparations. In a new paper, Do Civilian Complaints Against Police Get Punished?, DeAngelo (Claremont Colleges) et al. pursue this question empirically.

To do so, the authors examine arrest, complaint and charging data from Cook County (Illinois). The results indicate that a positive, causal link between citizen complaints and an increased number of total charges filed against a defendant exists. The paper also notes that a civilian complaint yields a larger plea discount, likely to induce a plea agreement. Notably, the main findings are entirely driven by charges filed directly by the prosecutor’s office rather than directly by the police. Given that prosecutors are likely filing charges after civilian complaints have been filed against law enforcement, it appears that prosecutor behavior is leading to upcharging in response to civilian complaints and not police decisions. An excerpted abstract follows.

“… This paper empirically investigates one way in which criminal justice institutions respond to civilian complaints. Namely, criminal prosecutors can upcharge a defendant who files a civil complaint against law enforcement. By upcharging, the prosecutor can increase the likelihood that a defendant will accept a plea deal, thus preventing the defendant from seeking monetary damages in civil court (Heck vs. Humphrey, 1994). Using data on citizen complaints and criminal charge outcomes from Cook County (Illinois), we find a strong causal link between a citizen filing a complaint and the total number of charges filed.”

The Empirics of Sobriety Checkpoint Stops

Motor vehicle crashes remain the leading cause of death for Americans under age 25, and many of these crashes involve substance use. To help reduce the volume of impaired drivers, many--but not all--states allow for sobriety checkpoints (SC). While historically checkpoint stops were used to enforce drunk driving laws, they are increasingly being used to deter and prosecute drugged drivers, especially in states where marijuana is legal.

Although “no-cause” traffic stops remain controversial, comparatively little data exist that helps inform related debates. While the SCOTUS permits such stops as a general matter, courts in eleven states have ruled that state constitutional provisions preclude sobriety checkpoints. And those states that discontinued sobriety checkpoints provide a critical source of data that a recent paper, Sobriety Checkpoint Law, Fatal Car Crashes and Arrests, exploits. In their paper, Lauren Jones (OSU—public affairs) and C. Blain Moran (OSU—public affairs) compare trends in event frequencies “across states that did and did not turn off their SC laws, net of confounding factors.” The paper finds, at bottom, that when states make sobriety checkpoints illegal, annual fatal crashes and DUI arrests increase. The paper also notes a racial effect relating to DUI and cannabis arrests. The abstract follows.

“Sobriety checkpoints have been used since the early 1980s to enforce laws prohibiting driving while intoxicated (DWI), and to deter DWI. However, their constitutionality has been questioned, as they allow police to stop drivers without cause. As a result, eleven states have banned their use. Using a two-way fixed effects event study analysis, we estimate the effect of rendering sobriety checkpoints illegal on state-level counts of fatal car crashes, and arrests for driving under the influence of drugs or alcohol (DUI), and drug possession. We find evidence that when states make sobriety checkpoints illegal, the number of annual fatal crashes increases by 17 percent. We also find evidence that rendering sobriety checkpoints illegal leads to a 25 percent increase in the number of annual DUI arrests in a state, suggesting that the deterrent effect of checkpoints is much stronger than their enforcement impacts. We also find suggestive evidence that eliminating sobriety checkpoints increases racial disparities in DUI and cannabis possession arrests, with larger impacts on arrests among Black residents.”

Theory's Role in ELS

A not infrequent, though diminishing, canard is that "too many" empirical legal studies papers are "atheoretical." While the claim itself is an empirical one (and one that I side-step), a larger, broader point endures. Specifically, the best of ELS is certainly informed by well-developed theory. In a recent paper, The Role of Theory in Empirical Legal Studies, Tonja Jacobi (Emory) makes this case in a brief and widely accessible paper. While perhaps targeted to "early career" ELS scholars and non-technical consumers of empirical research, the paper helpfully reminds all of theory's essential role in empirical legal studies. The paper's abstract follows.

"The growing field of empirical legal studies is not an alternative to theory: within every empirical inquiry is an implicit formal theoretical model. Further, good empirical tests should be inspired by substantive theories of the phenomena being analyzed. This essay explains how theory is built into all empirical testing and illustrates why empirical tests are best informed by well-developed theories. It uses examples of empirical studies of both US Supreme Court and Australian High Court oral argument to illustrate how these issues apply to the central issues at the intersection of empirical legal studies and constitutional law: the role of judicial ideology and strategy, and predicting judicial behavior. The essay explores common errors in both conducting and consuming empirical work, and presents some solutions, including both methodological fixes and how to draw on substantive expertise. It aims to help the non-technical reader understand when counterintuitive results should be queried and when, in contrast, resistance is more reflective of priors of the reader rather than potential weaknesses of the work. And it acts as a reminder to technical readers and empirical practitioners that theory is built into every empirical test, and so must be taken seriously."

CELS 2023--Univ. of Chicago Law School

Coming on the heels of an exceptionally well-executed and successful CELS at Virginia, CELS 2023 co-organizers--Adam Chilton, Dhammika Dharmapala, and William Hubbard--announced that CELS 2023 is scheduled for 13-14 October 2023, at the University of Chicago Law School. More information will follow.

Case Outcomes & Judge Financial Conflicts

To assist with insuring that "justice is blind," a presiding judge is expected to recuse from any case where he/she is financially conflicted with any of the litigating parties. Evidence that this expectation is not being consistently met was widely reported in 2020-2021. Indeed, the public controversy elicited a response from Chief Justice John Roberts. In his 2021 annual report on the federal judiciary, and in direct response to the controversy surrounding numerous federal judges, Chief Justice Roberts downplayed the controversy’s significance, characterizing the incidents as "isolated violations" and "unintentional oversights."

Notwithstanding Justice Roberts' characterizations, given the widespread implications posed by judges' failures to recuse when proper it is important to understand whether these recent episodes are one-time incidents or point to a larger, systemic problem. To address this question, in a recent paper, Judge Financial Holdings and Case Outcomes: Evidence from Judge Financial Disclosures, Tuhin Harit (Univ. Dallas--Mgmt.) et al. examine whether there is a relation between judges' disclosed financial holdings and the outcomes of the trials over which they presided.

Drawing from a sample that includes cases from 2000–2021, the paper finds that “stock ownership by judges significantly improves the chances of a litigant's winning a case.” In cases where the presiding judge is financially conflicted with either the defendant or plaintiff, “their chance of a favorable decision improves by 3.3% and 4.3%, respectively, after controlling for judge, industry, district, case file year, and firm characteristics.” An excerpted abstract follows.

"... Using novel data on financial disclosures of judges and the civil case information of public firms across district courts in the United States between 2000 and 2021, we document multiple instances in which judges fail to recuse themselves in cases where they hold common stock. We find robust evidence that judges rule more favorably and take longer when deciding over conflicted cases as compared to unconflicted ones. We also find that investors are positively surprised by these outcomes, as reflected by stock returns to trial outcomes. Collectively, our results provide the first large-scale evidence of the relation between judges' financial holdings and case outcomes.”

Dodd-Frank, Accounting Fraud, and Whistleblowing

Owing to several notable financial frauds involving whistleblowers, regulators took steps to strengthen whistleblowing provisions. One prominent example is the Dodd-Frank whistleblower program which provides financial rewards to whistleblowers who report financial fraud to the SEC. While regulators may believe that the Dodd-Frank whistleblower law helps deter accounting fraud, whistleblower tips can be frivolous and processessing such tips is not cost-less. Similar to all too many public policies, little empirically is known about the the Dodd-Frank whistleblower provision's efficacy.

In a recent paper, Did the Dodd-Frank Whistleblower Provision Deter Accounting Fraud?, Philip Berger (Chicago--Booth Business) & Heemin Lee (Baruch College) examine whether the Dodd-Frank whistleblower law has a causal effect on the deterrence of fraud and quantify the size of the deterrence effect. Exploiting a control group of firms previously exposed to state-level false claims statutes, the paper sets out to assess the "underlying (or ex ante) likelihood of accounting fraud decreases for firms subject to Dodd-Frank and not previously exposed to the risk of whistleblowing" via analogous state laws. At least as it relates to Dodd-Frank's whistleblowing component, the paper a finds a decrease in the probability of fraud attributable to whistleblower provisions. The abstract follows.

“We examine the deterrence effect of the Dodd-Frank whistleblower provision on accounting fraud. To facilitate causal inference, we use state False Claims Acts (FCAs), under which whistleblowing about accounting fraud at a firm invested in by a state's pension fund can result in monetary rewards from that state’s government. We divide our sample into firms exposed and not exposed to whistleblowing risk from a state FCA during the 2008 – 2010 period that preceded the 2011 SEC implementation of the Dodd-Frank whistleblowing provision. We hypothesize that firms already exposed to a state FCA whistleblower law are less affected by the Dodd-Frank whistleblower provision. Using the companies exposed to a state FCA as control firms in our Dodd-Frank tests, the remaining firms constitute the treatment sample. We find that exposure to Dodd-Frank reduces the likelihood of accounting fraud of treatment firms by 12% to 22% relative to control firms, but do not find that it affects audit fees.”

Med Mal Insurance Premia & Economic Theory

The medical malpractice insurance market is inherently complex and made even more so by the growing number of state efforts to regulate it. Complexities aside, however, standard economic theory should, in the main, explain, to some degree, med mal premia trends. Findings from a recent paper, How Do Insurers Price Medical Malpractice Insurance?, by Bernie Black et al. (Northwestern), however, "are not easily explained by standard economic theory."

Prof. Black et al.'s paper analyzes almost three decades worth of med mal premia data (gathered by the Medical Liability Monitor). Core findings include:

• Med mal premia are only loosely connected to insurer costs, even over long periods;
• Premia are much higher in states with damage caps, even after controlling for costs; and
• The Premium/Cost Ratio varies widely both across states at a given time and within states across time.

Taken together, the paper's findings imply that med mal insurance companies do not fully adjust premia to changes in direct costs even over long time periods as standard economic theory might predict.

Poisson Tutorial Videos

The good folks at Stata recently posted a series of video tutorials focusing on how to fit various Poisson regression models which assist when modeling count data. The various videos take up a Poisson model with: a) continuous predictors (here); b) categorical predictors (here); and c) Poisson models that include both continuous and categorical variables (here).

4th annual Association for Interdisciplinary Metaresearch and Open Science Conference (AIMOS2022) -- Registration

At the request of an ELS colleague, I want to help circulate the following registration information for the 4th annual Association for Interdisciplinary Metaresearch and Open Science Conference (AIMOS2022). Please note that the conference is scheduled for November 28-30, 2022, at the University of Melbourne. AIMOS2022 will be an in-person conference, with support to attend virtually.

As an association, AIMOS aims to make the research process more trustworthy and efficient and to promote the study of how research is done and how it can be improved.

The conference program will also explore other areas of metaresearch including a host of mini-note sessions on:

• Big Team Science
• Credible Science for Decision Making
• Correcting the Scientific Record
• History of Metascience 
• Unpacking Trust in Science
• The Many-Analysts EcoEvo Project
• The Systematizing Confidence in Open Research and Evidence (SCORE) Program.

In addition to invited speaker sessions, submissions are now open for: discussions about norms, practices, and cultures with science and scholarship more broadly; learning and development through practical skills workshops in open science (e.g., R, lab notebooks, pre-registration); and getting things done in hackathons.

Finally, a small number of AU$1,000 travel grants are also available to support in-person attendance.

A Small--But Important--Logistical Matter Incident to Dataset Merging

Those who have merged (or appended or crossed) datasets may have inadvertently bumped into a not uncommon problem--where the two separate datasets share a variable with the identical variable label name. For relatively smaller datasets, and where the researcher is aware of this hurdle ex ante, one can always, by hand, rename variables prior to combining the datasets. For larger datasets with many more variables, however, while this may not be practical it nevertheless remains critical.

A recent discussion of how to manage this nettlesome problem is found here. The comments' leading suggestion references Daniel Klein's -elabel- package, available from SSC (as it involves user-written code). Put briefly, the elable command allows a user to add a suffix (e.g., "_1") to every variable name in "dataset1." An analogous manipulation to datast2 can add another unique suffix (e.g., "_2") to every variable name in dataset2. And so forth. The end result includes datasets with distinct variable label names that can then be combined.

Helpful Statistics Resources

On the better late than never front, I just stumbled across a wonderfully rich "one-stop" resource for statistics novices. Germán Rodríguez (Princeton) hosts a site (here) that includes tutorials for both Stata and R. As to the tutorials' scope, the Stata tutorial, e.g., ranges from a "gentle introduction" to coding examples for customizable tables. Many--especially students--may find this resource helpful.

Firearms' Preferences and an Empirical Analysis of Secondary and Tertiary Regulatory Consequences

Civilian (household) gun ownership levels continue rise along with efforts to regulate gun ownership and possession. Scholarly efforts in this space, while increasing, confront complex data, modeling, and research design challenges.

In Preferences for Firearms and Their Implications for Regulation, the authors, Sarah Moshary (UC Berkeley—Business) et al., steer clear of trying to identify causal links between gun ownership and crime or gun deaths. Instead, the paper "estimates the effects of policy on both the number and types of guns sold in the primary market, as well as the consumer surplus that accrues to gun owners from their purchases.”

Among the various policy counterfactual simulations that the paper explores include a policy that bans assault weapons. What the paper's findings imply is a consumer shift from assault weapon ownership to firearm ownership. This is so, the paper argues, because even though consumers who are in the market for handguns do not consider long guns at all, many consumers that consider purchasing a long gun, by contrast, "are also interested in buying a handgun." As the "vast majority" of gun violence involves handguns, however, this substitution effect, fueled by a regulatory ban, introduces complicating policy wrinkles. The paper's abstract follows.

“This paper estimates consumer demand for firearms with the aim of evaluating the likely impacts of firearm regulations. We first conduct a stated-choice-based conjoint analysis and estimate an individual-level demand model for firearms. We validate our estimates using aggregate moments from observational data. Next, we use our estimates to simulate changes in the number and types of guns in circulation under alternative regulations. Importantly, we find that bans or restrictions that specifically target ‘assault weapons’ increase demand for handguns, which are associated with the vast majority of firearm-related violence. We provide distributions of consumer surplus under counterfactuals and discuss how those distributions could be useful for crafting policy."

Checks on Prosecutorial Power

In light of the significant authority that prosecutors wield, we want them to be, at once, both accountable and independent. Obviously, these twin goals are structurally at odds with each other. "Electing prosecutors ensures that they are accountable, but it limits their independence. In contrast, appointing prosecutors increases independence, but it does so at the expense of accountability." In the U.S., 45/50 states elect local prosecutors "who bear the primary responsibility for bringing felony prosecutions in their jurisdictions." However, as Carissa Hessick (UNC) et al. note in a recent paper, Understanding Uncontested Prosecutor Elections: "[prosecutor] elections in the United States generally fail to capture the accountability benefits of elections because so many prosecutor elections are largely uncontested and uncompetitive."

Drawing partly on data gathered by the Prosecutors and Politics Project, the paper models "contested" prosecutor elections. In a slight break from prior findings from related literatures, this paper's core findings emphasize the salience of whether an incumbent is running and the jurisdiction's population. The paper's (excerpted) abstract follows.

"... Using a logistic regression analysis, this Article estimates the individual effects of a number of variables on prosecutor elections. It finds that several factors that have been previously identified as contributing to uncontested election are not, in fact, what drives uncontested elections for prosecutors. Instead, the factors with the largest effect are whether an incumbent runs and the population of the district. It also identifies two features of state election law that contribute to the dearth of contested elections. The Article concludes by noting that these factors suggest specific policy changes that could help to increase the number of contested and competitive elections—thus ensuring that voters can help guide important criminal justice decisions in their communities."

The Empirics of Choice of Law

In an effort to peer into judicial decisionmaking more generally, a recent paper by Dan Klerman (USC), Bias in Choice of Law: New Empirical and Experimental Evidence, levers both observational and experimental data drawn from the vehicle accident context to examine choice of law disputes. The paper considers three leading sources of choice of law bias: pro-resident, pro-plaintiff, and forum law. The findings intrigue and complicate prevailing conventional wisdoms.

Specifically, core findings include that while judges seem to favor the plaintiff, judge decisions do not tilt in favor of application of the law they are most familiar with (the law of the forum). While state court judges favor local residents, federal court judges do not. Although the traditional “place of the injury” choice-of-law rule was more constraining than modern American choice-of-law standards (such as the Restatement Second), even judges under the traditional rule favored the plaintiff more often than seemed warranted. The abstract follows.

"Scholars have argued that modern American choice-of-law is subject to three biases: (1) a bias in favor of residents of the forum state (pro-resident bias), (2) a bias in favor of plaintiffs (pro-plaintiff bias), and (3) a bias in favor of the law of the forum (forum law bias). This article brings new evidence about these biases from a comprehensive database of vehicular accident cases and from the first experiment related to choice of law. Overall, there is some evidence of a pro-resident bias in state (but not federal) court, some evidence of pro-plaintiff bias in both federal and state court, and no evidence of a pro-forum bias."

Assessing Primary and Secondary Effects of Sobriety Checkpoints

While police use of sobriety checkpoints has become increasingly common since the 1980s, legal discomfort, particularly Fourth Amendment concerns, persist. While the SCOTUS concluded in its Sitz decision (1990) that sobriety checkpoints' social benefits outweighed costs on individual liberties, states remain free to preclude their use. Indeed, 11 states have ruled sobriety checkpoints illegal.

Aside from their legal status, empirical questions about sobriety checkpoints' efficacy lurk. In an intriguing new contribution to the emerging empirical literature, Sobriety Checkpoint Laws, Fatal Cars Crashes and Arrests, Lauren Jones (OSU--Public Policy) and C. Blain Morin (OSU--Public Policy) explore the effects of a state removing sobriety checkpoints on fatal car crashes and arrests.

Leveraging data from the Fatality Analysis and Reporting System and Uniform Crime Reports, the authors execute a twoway fixed effects (TWFE), event study analysis. As the paper notes, the twoway fixed effects analysis extends a difference-in-difference specification, where changes in outcomes across pre-treatment and post-treatment time periods are compared, across a treated unit and a control unit. As the abstract (below) makes clear, the paper's key findings include an annual 17 percent increase in fatal crashes and a 25 percent increase in the number of DUI arrests in states that cease using sobriety checkpoints.

“Sobriety checkpoints have been used since the early 1980s to enforce laws prohibiting driving while intoxicated (DWI), and to deter DWI. However, their constitutionality has been questioned, as they allow police to stop drivers without cause. As a result, eleven states have banned their use. Using a two-way fixed effects event study analysis, we estimate the effect of rendering sobriety checkpoints illegal on state-level counts of fatal car crashes, and arrests for driving under the influence of drugs or alcohol (DUI), and drug possession. We find evidence that when states make sobriety checkpoints illegal, the number of annual fatal crashes increases by 17 percent. We also find evidence that rendering sobriety checkpoints illegal leads to a 25 percent increase in the number of annual DUI arrests in a state, suggesting that the deterrent effect of checkpoints is much stronger than their enforcement impacts. We also find suggestive evidence that eliminating sobriety checkpoints increases racial disparities in DUI and cannabis possession arrests, with larger impacts on arrests among Black residents.”

Law School Curriculum: Reform, Assessment, and Research Design

While age-old complaints persist about the basic structure of the first-year law school curriculum, few law schools have dared to experiment with meaningful reforms. Fewer still study these reforms coherently. The "Curriculum B" experiment at Georgetown Law School, as well as a recent paper by David Hyman et al. (Georgetown), Does the 1L Curriculum Make a Difference?, are important and interesting exceptions.

Since 1991, Georgetown Law School has offered an "alternative" Curriculum B along side a standard, more traditional first-year curriculum. Curriculum B was designed to provide “critical” perspectives on the law and respond to important changes in legal practice and theory in the years since the standard 1L curriculum was first developed decades earlier. At Georgetown, students elect into Curriculum B and, when demand exceeds supply, student assignment to Curriculum B is determined by lottery.

For assessment purposes, the quasi-randomized control trial aspect of student assignment to Curriculum B provides a way to overcome selection bias. That is, Hyman et al. compare student outcomes for those who opted into Curriculum B (the treatment group) with student outcomes from those students who wanted to receive the "treatment" (Curriculum B) but, due to excess demand and random selection (lottery), were instead assigned the "placebo" (that is, to the standard, traditional first-year curriculum).

Aside from careful attention to research design, a few of the paper's warrant note. When it comes to students' overall law school academic performances, for example, the authors find that "Curriculum B leads to a mean preserving contraction in students’ overall grades. At the same time, we find that it leads to diminished performance in two key business law electives (Corporations and Securities Regulation)."

Business Litigants and the Roberts Court

Assessing “How pro business is the Roberts Court?” is both straightforward and complicated. Based on how business litigants fare in the Roberts Court versus non-business litigants over time, business does quite well. The win rate for business in the Roberts Court, 63.4 percent, is 15 percentage points higher than the next highest rate of business wins over the past century (the Rehnquist Court, at 48.3 percent). While it may be tempting for many to conclude that the current pro business tilt is purely a function of a Republican majority of justices on the Roberts Court, as Lee Epstein (USC) and Mitu Gulati (Virginia) argue in a recent paper, A Century of Business in the Supreme Court, 1920-2020, “this is only the beginning of the story.”

Epstein and Gulati bring data from the Supreme Court Database to the question. Starting with the 11,121 orally argued cases resulting in a signed opinion or judgment between the 1920-2020 terms, their paper focuses only on those 4,070 cases (35,173 votes) in which business was the petitioner (appellant) or respondent (appellee). That is, they excluded cases in which business was on both sides of a legal dispute.

Their core findings defy conventional wisdom and inject important complicating wrinkles. What they find includes: "(a) It is not just the Republicans on the Roberts Court who are more pro business than in prior Courts, but the Democrats as well; (b) The Government, through the SG’s office and across both Democratic and Republican administrations, has been much more supportive of business positions than in prior eras; and (c) An elite Supreme Court bar has emerged in recent years and businesses have hired them disproportionately so as to better influence the Court."