Empirical Legal Studies

Federal Judicial Data: Greater Transparency or Opacity?

While not itself an empirical paper per se, The Necessary and Proper Stewardship of Judicial Data, by Zachary Clopton (Northwestern) and Aziz Huq (Chicago), engages in a discussion near-and-dear to most legal empiricists' heart. Specifically, their paper sets out to develop "the first comprehensive descriptive, doctrinal, and normative analysis of judicial data.”

As the authors note, judicial (especially Art. III courts) data are "valuable in governance and for those interested in understanding how they are being governed. In the absence of hurdles or walls, it can illuminate the operation of one branch of the national government. Constitutional lapses can be brought to light through retail or wholesale data." As such, data flowing from Art. III courts and institutions can promote various public goods by, in part, advancing a better understanding of, or improvements to, judicial institutions.

One critical anomaly is that, as compared to the legislative and Executive branches, judicial branch data remain "largely terra incognito for scholars and the general public alike. Instead, public attention rushes to high-profile cases of leaked Supreme Court opinions and private influence campaigns.” While why this is so remains unclear (to me, anyway), that it is so, however, is obvious to any empirical legal scholar who labors and writes in this space. And the stakes incident to the "transparency v. opacity" debate grow more acute as new scholarly tools emerge (e.g., natural language processessing, text-scrapping).

As the authors make clear, under current policy and practice judicial data "might be lost, dissipating into air, or kept under wraps, never yielding up their secrets." In contrast, a commitment to greater transparency invites characterizing judicial data as a "new public asset that can then be tapped to create public goods for the benefit of all."

Data on the Fourth Amendment's "Good Faaith" Exception's Application

While the Fourth Amendment is designed to protect criminal defendants against prosecutors' use of unlawfully obtained evidence, an important exception exists for evidence obtained by law enforcement officers who rely in good faith on existing legal authority. Thus, if police rely on a statute that turns out to be unconstitutional or invalid warrants or precedent, the evidence obtained is nonetheless admissible under the good faith exception.

Empirical analyses on how the good faith exception works "on the ground," however, are scant. To this end, a recent paper, The Reality of the Good Faith Exception, by Matthew Tokson (Utah) and Michael Gentithes (Akron), helps to fill a void.

Their paper draws from available published (and unpublished) judicial decisions (N=5,531) that mention the Fourth Amendment or "motion to suppress" across three separate months (July 2015, 2018, and 2021). Among the paper's findings includes a discernible uptick in the exceptions salience over time (see Figure 1, below).

Figure 1: Good Faith Exception (GFE) Prevalence Over Time - Proportion of Fourth Amendment Suppression Cases Involving the Good Faith Exception

The authors are correctly mindful that selection effects certainly influence their descriptive findings. The authors argue that selection effects in this context likely lead, on balance, to an under-counting of suppression hearings to the extent that "defendants and their attorneys likely decline to file suppression motions because of the good faith exception." (emphasis added)

Overall, the paper argues that "the [good faith] exception, originally confined to narrow circumstances, is turning into a protection for nearly any investigatory activity with an arguable connection to an existing precedent or statute" and that "[c]ourts rely on the exception with remarkable frequency, and a large proportion of courts invoking the exception avoid any substantive constitutional ruling, stunting the development of Fourth Amendment doctrine."

Experimental Judicial Decisionmaking Studies & External Validity

For experimental judicial decisionmaking studies, the "gold-standard" research design includes using actual judges as participants. My Cornell colleague (and good friend) Jeff Rachlinski is a leading scholar in this space. For an array of reasons (including practical reasons), too much of the experimental research in this field is hampered by researchers' use of students (including law students) as the experimental participants. To the extant that researchers seek to generalize results from judicial decisionmaking studies to the "universe of judges," conventional  wisdom suggests that the use of students in these experiments raises important external validity questions.

Of course, such (and most) "conventional wisdoms" rest on empirical assumptions. And in a recent paper, Can Law Students Replace Judges in Experiments of Judicial-Making?, Holger Spamann (Harvard) and Lars Klohn (Humboldt) submit this question to an experiment. The answer to the question they pose in their paper's title?: "unfortunately, no." The paper's abstract follows.

"Experimental research on judicial decision-making is hampered by the difficulty of recruiting judges as experimental participants. Can students be used in judges’ stead? Unfortunately, no. We ran the same high-context 2×2 factorial experiment of judicial decision-making focused on legal reasoning with 31 U.S. federal judges and 91 elite U.S. law students. We obtained diametrically opposed results. Judges’ decisions were strongly associated with one factor (sympathy, i.e., bias) but not the other (law). For students, it was the other way around. Equality between the two groups is strongly rejected. Equality of document-view patterns—a proxy for thought processes—and written reasons is also strongly rejected."

Modeling Rare Events--SCOTUS Clerkships

Of the approximate 36,000 law school students who graduate each year, only 36 or so end up securing a SCOTUS clerkship. While a handful of biographical and anecdotal accounts exist, very few systematic data exist on the "1-in-1,000" clerkship lottery.

A recent paper by Tracey George (Vanderbilt), Albert Yoon (Toronto), and Mitu Gulati (Virginia), Some Are More Equal Than Others: U.S. Supreme Court Clerkship, digs into 40 years worth of data and provides rich granularity about this statistical rare event. In the main, while some unexpected twists emerge, what the authors find generally comports with perceived wisdoms. For example, a disproportionate number of clerks emerge from five elite law schools. One interesting change, however, is that while a SCOTUS clerkship once led "primarily to government and universities [law faculties]," former clerks now increasingly land at "law firms and corporations." An excerpted abstract follows.

"... Yet, we lack a clear story about who wins this clerkship lottery. This Essay seeks to provide that story. Our analysis relies on new datasets of all Court clerks who served between 1980 and 2020, including the details of their path to the Court and their road after. We amend and expand on theories of success in this important labor market. All clerks are highly qualified, yet the same is true for many not chosen. We find that educational pedigree, as opposed to academic performance or any other qualification, often distinguishes the winners from the also-rans. The Court clerkship selection process, like other elite labor markets, proves to be a blend of status and merit where status often prevails. Our analysis does not end there, however. We go on to look at where this forty-year cohort is currently working and confirm that once attained, a Court clerkship does lead to a bounty of opportunities including a return to the Court as a justice. Thus, the Court clerkship lottery is an important labor market not only to lawyers but also to society writ large. In the elite legal labor market, some people are, in fact, more equal than others."

Theory's Role in ELS

A not infrequent, though diminishing, canard is that "too many" empirical legal studies papers are "atheoretical." While the claim itself is an empirical one (and one that I side-step), a larger, broader point endures. Specifically, the best of ELS is certainly informed by well-developed theory. In a recent paper, The Role of Theory in Empirical Legal Studies, Tonja Jacobi (Emory) makes this case in a brief and widely accessible paper. While perhaps targeted to "early career" ELS scholars and non-technical consumers of empirical research, the paper helpfully reminds all of theory's essential role in empirical legal studies. The paper's abstract follows.

"The growing field of empirical legal studies is not an alternative to theory: within every empirical inquiry is an implicit formal theoretical model. Further, good empirical tests should be inspired by substantive theories of the phenomena being analyzed. This essay explains how theory is built into all empirical testing and illustrates why empirical tests are best informed by well-developed theories. It uses examples of empirical studies of both US Supreme Court and Australian High Court oral argument to illustrate how these issues apply to the central issues at the intersection of empirical legal studies and constitutional law: the role of judicial ideology and strategy, and predicting judicial behavior. The essay explores common errors in both conducting and consuming empirical work, and presents some solutions, including both methodological fixes and how to draw on substantive expertise. It aims to help the non-technical reader understand when counterintuitive results should be queried and when, in contrast, resistance is more reflective of priors of the reader rather than potential weaknesses of the work. And it acts as a reminder to technical readers and empirical practitioners that theory is built into every empirical test, and so must be taken seriously."

Empirical Contours of the Political Question Doctrine

Amid the enduring and varied debates about the political question doctrine, motivated by the Supreme Court's 1962 Baker v. Carr decision, the scholarly commentary shares one key attribute. Specifically, it dwells on the Supreme Court. Such a focus is ironic as while the Court has only rarely applied the political question doctrine, lower federal courts, by contrast, have invoked the doctrine "as a basis for dismissal in hundreds of cases." Drawing on both qualitative and quantitative (descriptive) methods, in a recent paper, The Real Political Question Doctrine, Curtis Bradley (Chicago) and Eric Posner (Chicago) explore how the political question doctrine has operated in the lower courts since the Supreme Court's Baker decision.

The authors find that lower courts use the doctrine to "evaluate their own institutional capacity to resolve politically sensitive disputes. It is the lower courts’ more limited capacity compared to the Supreme Court that explains the lower courts’ heavier reliance on the doctrine." More specifically, "the political question doctrine is a screening mechanism that the lower courts use to take account of limits on their institutional capacity. The Supreme Court has less need for this mechanism, so most of the doctrine’s life must be found in the foothills rather than at the summit of Mount Olympus. From this standpoint, the doctrine is not moribund but full of life. It is not limited in the ways suggested by the 'classical' or 'functional' theories. The lower court practice is best captured by Bickel’s prudential theory." An excerpted abstract follows.

"... We provide the first empirical account of how the doctrine has operated in the lower courts since Baker. Our account is based on both a quantitative analysis of a sample of these decisions and on a qualitative review of these and other decisions. This account reveals a political question doctrine that is substantially different from the one described in most scholarship: it is more vibrant, heavily focused on foreign affairs, often applied in non-constitutional cases, more prudential, and not a permanent disallowance of judicial review...."

Mistaking Correlation for Causation

Economists--and econometrics--have been involved in empirical legal studies for generations. This is particularly true for anti-trust doctrine, both its development and current application. Legal challenges to proposed mergers, and questions relating to anti-competitive spill-over effects, including market concentration, remain a staple for empirical analyses.

In the anti-trust context, one intuitive move involves assessing whether "prices are positively related to measures of concentration, such as the Herfindahl-Hirschman Index (HHI), comparing across different geographic markets or time periods." Pursuing such a question can involve (and frequently does involve) something as simple (and potentially powerful) as regressing price on the HHI.

Despite whatever intuitive appeal such a move holds, in a recent brief essay--co-signed by many leading law and economics scholars (including, and I add in interest of full disclosure, my Cornell colleague George Hay), On the Misuse of Regressions of Price on the HHI in Merger Review, the authors caution against such a move as it invites one basic fundamental problem: Mistaking correlation for causation. And it is for the essay's description of just how such a basic mistake can arise (rather than it's specific application in the anti-trust context) that I recommend this essay, especially to younger, developing empirical legal scholars.

As it relates to the specific anti-trust point developed in the essay, the authors' argue that: "The underlying problem with regressions of price on the HHI is that the relationship between price and the HHI is not causal. Instead, both are equilibrium outcomes that are determined by demand, supply, and the factors that drive them. Thus, a regression of price on the HHI does not show the sort of causal effect that would be helpful in predicting the competitive effects of a merger." More specifically, as the "HHI can take values ranging from 0 to 10,000, with the former corresponding to a fragmented market with infinitesimal firms, and the latter to monopoly. All else equal, a decrease in the number of firms leads to a higher HHI. However, the HHI can increase if, for a given number of firms, the market shares of the larger firms increase."

The root problem, of course, flows from challenges posed by “endogeneity” or “simultaneity.” Such problems lurk "when a clear causal path between two variables is posited by the relevant economic theory but the causal factor (on the right side of the regression equation) is either correlated with unobserved factors or, more subtly, is simultaneously determined by the variables (in this case, price) on the left side of the regression. A famous example of simultaneity involves price and quantity, which are simultaneously determined in the classic economic model of supply and demand." The problems posed by "endogeneity” or “simultaneity," of course, are not limited to the anti-trust realm. An excerpted abstract follows.

"It might seem natural to determine whether prices are positively related to measures of concentration, such as the Herfindahl-Hirschman Index (HHI), comparing across different geographic markets or time periods. This might be implemented by using a simple regression of price on the HHI. However, for reasons that we describe, regressions of price on the HHI should not be interpreted as establishing causation. That is, they do not inform how a change in concentration from a merger would affect prices. Courts and other policy-makers should not rely on regressions of price on the HHI for the purposes of antitrust merger review."

A Descriptive Glance at Police Clearance Rates

While I have an admitted affinity for asking simple questions, sometimes it's the "simple" question that can help point the way toward more sophisticated empirical work. A recent paper by Andrew Leipold (Ill.), The Puzzle of Clearance Rates, and What They Can Tell Us About Crime, Police Reform, and Criminal Justice, makes my point.

The paper begins by framing its core research question: "Recent incidents of police violence have led to widespread reform efforts, from modest proposals to change police practices to dramatic attempts to slash funding or abolish the police entirely. But largely ignored in the debate is a simple question – how well is law enforcement currently performing its core functions? In particular, how good are the police at finding the perpetrator, arresting that person, and gathering enough evidence to start the matter through the criminal justice system?"

Exploring police clearance rate data engages the question. And what Leipold finds is that "Clearance rates for violent and property crimes have been both quite low and amazingly steady for the last 40 years. These figures are counterintuitive, because during that same period, crime first rose and then decreased dramatically; law enforcement personnel numbers increased, and then flattened; and the legal enforcement landscape appears to have tilted in the direction of the police and prosecution. Each of these changes should have significantly affected the clearance rates, but even collectively, they did not."

The presentation of basic descriptive trend data amply supports Leipold's point. And what he argues is that "[w]hile there is no single explanation, the diversion of resources from index crimes and a change in focus toward crime prevention seem to have the most explanatory power." While how to interpret--or explain--the counterintuitive trends, of course, remains ripe for future work, part of the power of initial descriptive work is that it helps bring into sharper relief avenues for deeper and more methodologically sophisticated future research.

"Judging" Autonomous Vehicle Liability

Conventional wisdom implies that, on balance, judges, as a group and in their professional capacity, tend to disfavor new technologies. This judicial lurch towards disfavoring can present important policy implications as, especially in common law systems, judges can play a critical role in the regulation of new technologies. One example that will likely play out soon in real-time involves autonomous cars.

Dominating the experimental judicial decisionmmkaing literature are studies that rely on "mock" judges deciding "experimental" case hypotheticals. All too often, however, these "mock" judges involve either undergraduate students or a non-random collection of MTurk respondents. By contrast, findings from a recent paper, Judging Autonomous Vehicles, by Jeff Rachlinski (Cornell) and Judge Andrew Wistrich (Ca Central Dist. Ct), both valued Cornell colleagues, reflect an important advancement insofar as they derive from actual, real-live sitting judges (N=933).

In their studies, the authors presented hypothetical cases in which judges had to assess fault or determine damages for either an accident caused by an autonomous vehicle or by a human driver. What the paper finds is that "judges will treat automated vehicles with skepticism. In the related experiments we conducted, judges treated an autonomous vehicle that had been involved in a car accident more harshly than a human-driven vehicle involved in an essentially identical accident.” As these results comport with others prior research focusing on other technologies, larger policy implications loom. As the authors note, "at least when an innovation is artificial, non-human, and a dramatic departure from existing practices, judges act as a brake, delaying and hindering its adoption." The paper's abstract follows.

“The introduction of any new technology challenges judges to determine how it into existing liability schemes. If judges choose poorly, they can unleash novel injuries on society without redress or stifle progress by overburdening a technological breakthrough. The emergence of self-driving, or autonomous, vehicles will present an enormous challenge of this sort to judges, as this technology will alter the foundation of the largest source of civil liability in the United States. Although regulatory agencies will determine when and how autonomous cars may be placed into service, judges will likely play a central role in defining the standards for liability for them. How will judges treat this new technology? People commonly exhibit biases against innovations such as a naturalness bias, in which people disfavor injuries arising from artificial sources. In this paper we present data from 933 trial judges showing that judges exhibit bias against selfdriving vehicles. They both assigned more liability to a self-driving vehicle than they would to a human-driven vehicle and treated injuries caused by a self-driving vehicle as more serious than injuries caused by a human-driven vehicle.”

Data on Insider "Giving"

When corporate insiders learn about material, non-public bad news at their company that devalues stock prices, they can avoid losses by selling their stock before the public finds out. This form of “insider trading,” is, of course, illegal and risky. But selling stock is not the only way for corporate insiders to cash out. One alternative strategy, which is almost as financially effective and benefits from comparatively less onerous reporting requirements and legal restrictions, involves donating the stock to a charity and taking a tax deduction at the stock's inflated price. While a financial "second-best" strategy, "insider giving" nonetheless endures as a plausibly potent substitute for insider trading.

In Insider Giving, Sureyya B. Avci (Sabanci Univ.--Business) et al., submit their "insider giving" hypothesis to data. As the paper describes, the dataset includes "all reported gifts of common stock and contains over 9,000 observations between 1986 and 2020. The total volume of gifts contained in our dataset is approximately 2.1 billion shares, with a dollar value of approximately $50 billion." And what they find is that "large shareholders’ gifts are suspiciously well timed. Stock prices rise abnormally about 6% during the one-year period before the gift date and they fall abnormally by about 4% during the one year after the gift date, meaning that large shareholders tend to find the perfect day on which to give."

Rather than ascribe these finding to insider-donors' incredible good luck, the paper instead develops two possible alternative explanations: "information leakage" and "backdating." Specifically, "executives seem to provide large shareholders with material non-public information, who then use it to time gifts. We also find a second explanation to be supported, though its magnitude is smaller – backdating. The telltale sign of backdating is that the givers’ extraordinary luck tends to grow alongside the delay they take in reporting the gift. A donor who waits a few weeks to report a gift can cherry pick the very best date to retroactively claim their gift was consummated. That is precisely what we find." An excerpted abstract follows.

"Corporate insiders can avoid losses if they dispose of their stock while in possession of material, non-public information. One means of disposal, selling the stock, is illegal and subject to prompt mandatory reporting. A second strategy is almost as effective and it faces lax reporting requirements and legal restrictions. That second method is to donate the stock to a charity and take a charitable tax deduction at the inflated stock price. “Insider giving” is a potent substitute for insider trading. We show that insider giving is far more widespread than previously believed. In particular, we show that it is not limited to officers and directors. Large investors appear to regularly receive material non-public information and use it to avoid losses. Using a vast dataset of essentially all transactions in public company stock since 1986, we find consistent and economically significant evidence that these shareholders’ impeccable timing likely reflects information leakage. We also document substantial evidence of backdating – investors falsifying the date of their gift to capture a larger tax break….”

JELS' Research Notes and Datasets Section

If memory serves, a suggestion in 2016 (or so) from Bernie Black (Northwestern) and Marty Wells (Cornell-Statistics) to the JELS editors set into motion a chain of events that culminated in 2017 with the introduction of the "Research Notes and Datasets" section in JELS. The core motivation behind introducing a new paper "genre" for JELS was a fundamental belief that dataset construction, data sharing, and methods testing are a vital part of ELS' growth. We felt that a new section in JELS would help contribute to the realization of such beliefs. What we did not fully appreciate or anticipate, however, was the response. Since the Research Notes and Datasets section's launch in 2017 JELS has already published a number of important papers, including:

Patent Similarity Data and Innovation Metrics, by Ryan Whalen Alina Lungeanu Leslie DeChurch Noshir Contractor JELS 2020 (September). https://onlinelibrary.wiley.com/doi/10.1111/jels.12261

Developing High‐Quality Data Infrastructure for Legal Analytics: Introducing the Israeli Supreme Court Database, by Keren Weinshall Lee Epstein JELS 2020 (June). https://onlinelibrary.wiley.com/doi/10.1111/jels.12250

The German Federal Courts Dataset 1950–2019: From Paper Archives to Linked Open Data, by Hanjo Hamann Sokol JELS 2019 (September). https://onlinelibrary.wiley.com/doi/10.1111/jels.12230

Competition Law Gone Global: Introducing the Comparative Competition Law and Enforcement Datasets, by Anu Bradford Adam S. Chilton Christopher Megaw Nathaniel Sokol JELS 2019 (June). https://onlinelibrary.wiley.com/doi/10.1111/jels.12215

Accuracy and Fairness for Juvenile Justice Risk Assessments, by Richard Berk JELS 2019 (March). https://onlinelibrary.wiley.com/doi/10.1111/jels.12206

Text of Trade Agreements (ToTA)—A Structured Corpus for the Text‐as‐Data Analysis of Preferential Trade Agreements, by Wolfgang Alschner Julia Seiermann Dmitriy Skougarevskiy  JELS 2018 (September). https://onlinelibrary.wiley.com/doi/10.1111/jels.12189

Medical Liability Insurance Premia: 1990–2016 Dataset, with Literature Review and Summary Information, by Bernard Black Jeanette W. Chung Jeffrey Traczynski Victoria Udalova Sonal Vats, JELS 2017 (March). https://onlinelibrary.wiley.com/doi/10.1111/jels.12146

Submitting Bail Reforms to Data: Competing Interpretations in Cook County, Illinois

Various bail reforms are quickly emerging across jurisdictions. Proponents argue, at bottom, that such reforms will not impose undue increased risk on public safety. Such a claim is, of course, an empirical one. And, as is far too common in the “legal world,” such claims are made without the benefit of careful empirical study.

Enter bail reform initiatives in Cook County, Ill (functionally, Chicago) that were implemented in 2017 and studied in 2019. What has ensued—and is the focus of a recent paper by Paul Cassell (Utah) and Richard Fowles (Utah—econ), Does Bail Reform Increase Crime? An Empirical Assessment of the Public Safety Implications of Bail Reform in Cook County, Illinois—is a dispute over how to best interpret the study’s (that is, the “Bail Reform Study,” supervised by the Cook County Circuit Court) findings.

While the Bail Reform Study’s authors landed on far more “optimistic”  conclusions, the Cassell and Fowles paper, by contrast, has “serious doubts about its upbeat conclusions.” Their paper drills down on two main points. First, “the [Bail Reform] Study fails to recognize that, given that more defendants are being released after the reforms, even a 'stable' rate of community safety will inexorably lead to more crimes. That stable rate of safety—and, inversely, the stable rate of failure or public safety danger—applies across a larger pool of released defendants, which necessarily means that the public suffers additional crimes. In other words, at least in Cook County, more bail reform apparently means more crimes." Second, “we find that, contrary to the Study’s suggestion of stability, the number of crimes committed by pretrial releasees appears to have significantly increased.” A slightly excerpted abstract follows.

“Recently bail reform issues have been in the news across the country, as concerns about fair treatment of defendants and possible public safety risks from expanding pretrial release have collided. These issues involve important empirical questions, including whether releasing more defendants before trial leads to additional crimes. An opportunity to investigate this public safety issue has developed in Chicago, our nation’s third largest city. There, the Office of the Chief Judge of the Cook County Courts adopted new bail reform measures in September 2017 and reviewed them empirically in May 2019. Cook County’s Bail Reform Study concluded that the new procedures had released many more defendants before trial without any concomitant increase in crime. This article disputes the Study’s conclusions. This article explains that, contrary to the Study’s assertions, the new changes to pretrial release procedures appear to have led to a substantial increase in crimes committed by pretrial releasees in Cook County. Properly measured and estimated, after more generous release procedures were put in place, the number of released defendants charged with committing new crimes increased by 45%. And, more concerning, the number of pretrial releasees charged with committing new violent crimes increased by an estimated 33%. In addition, as reported by the Chicago Tribune, the Study’s data appears to undercount the number of releasees charged with new violent crimes; and a substantial number of aggravated domestic violence prosecutions prosecutors dropped after the changes, presumably because batterers were able to more frequently obtain release and intimidate their victims into not pursuing charges. These public safety concerns call into question whether the bail “reform” measures implemented in Cook County were cost-beneficial.”

Judicial Citation Patterns and NY County Surrogate's Courts

As ELS scholars continue to develop judge citation studies--typically from the "top-down" (that is, we know comparatively far more about judge citation patterns in the SCOTUS and other federal courts than non-federal judicial contexts)--comparatively far less is know about judge citation patterns in various lower and state court systems. A recent paper by Bridget Crawford (Pace), What Probate Courts Cite: Lessons from the New York County Surrogate's Court 2017-2018, contributes to a scholarly gap by describing the basic contours of judge citation patterns in New York County Surrogate's Court during 2017 and 2018. New York County's Surrogate's courts serve the most densely populated county in the United States. An excerpted abstract follows.

"... This Article makes three principal claims — one empirical, one interpretative, and one normative. This Article demonstrates through data, derived from a study of all decrees and orders issued by the New York County Surrogate’s Court in the years 2017 and 2018, that the probate court located in the most densely populated county in the United States cites fewer authorities less often than almost any other court (of any level) for which data is available.

There are a variety of factors that may explain this low rate of citation by the New York County Surrogate’s Court including docket size, the size and composition of the court’s staff, a judicial perception that the application of the law is a relatively mechanical process, or a subjective determination that speed in processing the court’s docket outweighs any public interest in citation-replete decrees and orders. Yet by increasing its engagement with a range of authorities, the New York County Surrogate’s Court (and indeed any probate court) may increase public confidence in the judiciary while also enhancing understanding of trusts and estates as a complex and dynamic area of law."

Everything You Ever Wanted to Know About Copyright Registrations Over Time

A recent paper reports results from the authors' effort to build what now has to be the single largest U.S. Copyright registration data set. In An Empirical Study of 225 Years of Copyright Registrations, Zvi Rosen and Richard Schwinn, report on what they find from observing more than two centuries of registration data. "This empirical analysis highlights two new insights that our analysis provides: The first is that the copyright term is, to a point, highly significant to registrations. In other words, a longer copyright term leads to more copyright registrations being filed. This is suggestive but not conclusive of a connection between a longer term and the creation of new works. Our analysis also shows that the incentive effect of a longer term offers diminishing returns above a certain point, and is no longer significant at a term of about 70 years. Secondly, we observe for the first time that the passage of a complete revision to the Copyright Law in 1909 which made registration (theoretically) optional without losing copyright protection represented a sharp paradigm shift for the relationship of copyright to economic progress.”

Equally interesting however, is the data set itself. As the authors note, “Until now, scholars have been dependent on limited tallies offered by the Annual Reports of the Register of Copyrights, which have provided the basis for all previous studies. In contrast, this study utilizes the more detailed statistics from the Catalog of Copyright Entries (“CCE”), which also have the benefit of corresponding to calendar year instead of the federal government’s fiscal year. This is also the first study ever to provide yearly estimates of copyright registrations per year prior to 1870 in any form – prior to this study the only figure used was 150,000 for the entire 80 year period… By taking account of changes in agency practice at the U.S. Copyright Office and its predecessors, we can harmonize our data over the entire series of 1790 to 2015.”

Gelman and Carlin's 'Type M' and 'Type S' Errors and Stata's rdesigni

Increasingly "fed up" the concepts of "'power,' 'type 1 error,' and 'type 2 error,' because all these are defined in terms of statistical significance," Andrew Gelman (Columbia--Statistics) developed and advanced "Type S (Sign) and Type M (Magnitude) Errors." Gelman, conceding that "concepts of Type S and Type M errors are not perfect," nonetheless concludes they represent a helpful "step forward." Gelman (with John Carlin) described his thinking in a 2014 paper, Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors. As well, the paper is summarized in a more recent blog post (here).

While few are ready to completely jettison more traditional power testing, Gelman and Carlin's alternative "Type S and Type M" struck many as a useful complement. Additional evidence of the possible promise of "Type S" and "Type M" tests is found in the recent release of Daniel Klein's user-written Stata command add-on, entitled rdesigni. (For some background on the new rdesigni command click here for a recent list-serv discussion.)

Those already equipped with Stata and wanting to explore this new command need only execute to following commend (with Stata launched):

ssc install rdesigni