Empirical Legal Studies

Balanced v. Unbalanced Panel Data; "reg" v. "xtreg" Commands--Asking the Right Question

An admittedly small but nonetheless important point emerged in a recent StataList discussion (here). While the original query wondered whether a decision between the "reg" and "xtreg" commands pivoted on whether panel data were balanced or unbalanced, a very helpful commenter quickly made clear the question's pivotal (and mistaken) assumption. To be sure, while the "reg" and "xtreg" commands may yield the same or similar results in certain conditions, they estimate different models and different parameters. In general, where panel data are involved (balanced or unbalanced), as the commenter notes,"-reg- is simply inappropriate."

CELS 2019 Submission Deadline--Extended to June 28, 2019 (Midnight, Pacific Time)

Claremont McKenna College, led by conference co-organizers Eric Helland, David Bjerk, and Daniel Krauss, and the Society for Empirical Legal Studies (SELS) are pleased to announce the 14th Annual Conference on Empirical Legal Studies (CELS) on November 15-16, 2019, in Claremont, California.

CELS is a highly regarded interdisciplinary gathering that draws researchers from across the United States and the world, bringing together scholars in law, economics, political science, psychology, policy analysis, and other fields who are interested in the empirical analysis of law and legal institutions. Papers are selected through a peer review process, and discussion at the conference includes assigned commentators and audience questions.

CELS will consider empirical papers spanning all areas of empirical legal studies. Authors are encouraged to submit works-in-progress; however, submissions should be completed drafts that include principal results. Submitted papers must be unpublished (and expected to be unpublished at the time of the conference). If accepted, authors will have an opportunity to submit a revised draft prior to the conference for presentation and discussion. Please note that accepted papers will be made available to all conference participants.

The paper submission deadline has been extended to June 28, 2019 (midnight, Pacific Time). To submit a paper or for more information about CELS 2019, including conference registration, please go to the CELS 2019 conference website (here).

Testing Police Officers' "Intuition" About "High-Crime" Areas

In a fascinating paper, The End of Intuition-Based High-Crime Areas, Ben Grunwald (Duke) and Jeffrey Fagan (Columbia) engage an important criminal procedure question with the benefit of a massive data set. The paper begins with the Supreme Court's conclusion in Wardlow that a "suspect’s presence in a 'high-crime area' is relevant in determining whether an officer has reasonable suspicion to conduct an investigative stop." Perhaps not surprisingly, the Court's opinion in 2000 provides little guidance about what constitutes a "high-crime" area. The paper sets out to assess how the doctrine was operationalized by examining data on more than two million investigative stops conducted by the NYC police from 2007 to 2012.

The data are especially rich in that in NYC police officers who conduct an investigative stop are required to complete a form after each one. "The forms collect rich information on suspect demographics and the precise geographic location of each stop. The data also contain anonymized officer identifiers, which allow us to observe how the same officer behaves in different areas, and how different officers behave in the same areas. And, most important for our purposes, the forms require officers to check off a series of roughly twenty boxes, indicating the bases of suspicion that justified the stop. Fortunately, one of those boxes is for high-crime areas. We merged this dataset with crime statistics and racial and demographic information on small geographic areas in New York City.”

As the paper makes clear, the data cast uneasy light on the Wardlow decision. "Our results suggest that Wardlow may have been wrongly decided. Specifically, we find evidence that officers often assess whether areas are high crime using a very broad geographic lens; that they call almost every block in the city high crime; that their assessments of whether an area is high crime are nearly uncorrelated with actual crime rates; that the suspect’s race predicts whether an officer calls an area high crime as well as the actual crime rate; that the racial composition of the area and the identity of the officer are stronger predictors of whether an officer calls an area high crime than the crime rate itself; and that stops are less or as likely to result in the detection of contraband when an officer invokes high-crime area as a basis of a stop. We conclude with several policy proposals for courts, police departments, and scholars to help address these problems in the doctrine.”

Equally interesting is that the authors point the way for fruitful follow-up empirical work on this topic. "In future empirical work, legal scholars can use body camera footage or SSO to validate the most common factors invoked by officers to establish reasonable suspicion. They can do so by assessing the factual accuracy of police officers’ claims about the presence of these factors and by examining whether they are in fact predictive of criminal behavior. Armed with that information, scholars and courts will have a clearer picture of what Fourth Amendment factors provide a reliable basis for reasonable suspicion and meaningful protection against unreasonable intrusions on personal privacy."

A New Legal Cloud Over Statistical Analyses of Judicial Decisions in France

In a move that will strike many court scholars (and, likely, others as well) as verging on the "unbelievable" (in the literal sense of the word), according to emerging news reports (e.g, here) the French Government has now "banned the publication of statistical information about judges’ decisions–with a five year prison sentence set as the maximum punishment for anyone who breaks the new law." The new french law (Art. 33 of France's Justice Reform Act) makes clear that "[t]he identity data of magistrates and members of the judiciary cannot be reused with the purpose or effect of evaluating, analysing, comparing or predicting their actual or alleged professional practices.” (translated; emphasis added)

According to news accounts, the new french law targets "legal tech companies focused on litigation prediction and analytics–from publicly revealing the pattern of judges’ behaviour in relation to court decisions." That said, the implications for empirical legal scholars seeking to model judicial decisions, certainly at the individual judge-level, in France are obvious.

While empirical legal scholars seeking to analyze U.S. judicial decisions face bothersome "hurdles" when it comes to linking judge names and their published decisions (e.g., a long-standing AO policy that strips judge names/IDs from most publicly-available AO data sets), I am unaware of anything in the U.S. that comes close to what the French Government seeks to accomplish with its new law.

Data on Immigration Judge Decisions Over Time

Public (and scholarly) debate over immigration laws and their enforcement all too often takes place in a data vacuum. A recent paper by Catherine Kim (UNC) and Amy Semet (Columbia), An Empirical Study of Political Control Over Immigration Adjudication, sets out to reduce this data paucity. Exploiting a large data set that includes every immigration removal proceeding decision from January 2001 through November 2018 (N=800,000+), the paper endeavors to isolate and assess the potential influence of an Administration's political preferences. The excerpted abstract, below, summarizes the paper's key findings.

"This Article presents the first comprehensive empirical assessment of the extent to which Immigration Judges (IJs), the administrative officials charged with adjudicating whether a given noncitizen will be deported from the country, decide cases on the basis of an Administration’s political preferences rather than an independent assessment of the legal merits of a given case. 

We constructed an original dataset of over 800,000 removal proceedings decided between January 2001 and November 2018. First, we found that every Administration—not just the current one—disproportionately appoints IJs with backgrounds in the former Immigration and Naturalization Service (INS), the Department of Homeland Security (DHS), or the Department of Justice (DOJ), agencies with responsibilities in prosecuting noncitizens. Second, using logistical regression to control for over a dozen variables that might impact a decision to remove, we found that the identity of the Administration that appointed an IJ was not a statistically significant predictor of the likelihood of ordering removal. That is, after controlling for other variables, Trump appointees were no more likely to order removal than Obama or Bush appointees.

Finally, using logistical regression and controlling for other variables, we found that the Administration in control at the time of decision is a statistically significant predictor of removal rates. For example, Bush appointees who served during the Bush, Obama, and Trump Eras were far more likely to order removal during the Trump Era. Specifically, when all other variables are held constant, they were 14% less likely to order removal during the Obama Era than during the current Era, and 12% less likely to order removal during the Bush Era than during the Trump Era. These results suggest that the current Executive-in-Chief exercises a profound influence over removal decisions, undermining the assumption of independence among administrative adjudicators."