Empirical Legal Studies

Stata Graphic "Trick" of the Day

For too many (myself included), Stata graphics can be among the more complicated coding tasks. Thus, concrete coding tips for some of the more "exotic" graphics capabilities invariably warrant attention (even if only to satisfy curiosity). In this spirit I pass along a brief discussion concerning coding requirements for a "fan" chart in Stata (here). In an example of what is possible (below), the presentation of a standard "line" chart is graphically enhanced with +/- 1, 2, and 3 standard error "cones."

Conference on Data Science and Law

Prof. Elliott Ash (ETH-Zurich) passed along information on a conference submission deadline extension for the Conference on Data Science and Law, hosted by ETH-Zurich, Switzerland, on 7-8 June 2019. The conference’s submission deadline is now: 12 Feb. 2019. An excerpted conference description and related information follow.

“Interest is growing among scholars from a variety of disciplines and across the globe in research at the intersection of data science and law. In recent years, the digitization of legal texts and developments in statistics and computer science have paved the way for new methodological approaches in this area…. The inaugural Conference on Data Science and Law will convene researchers across disciplines who are interested in this burgeoning field. Scholars in law, social and behavioral sciences, digital humanities, computer science, machine learning, and data analytics are invited to present works-in-progress in all areas of empirical legal studies that involve big data sources or use data science techniques, including natural language processing, machine learning, algorithmic fairness, topic modeling, or network analysis."

For those interested, paper proposals can be submitted directly to: dasa.skopp@gess.ethz.ch, with an email subject line: “Conference Submission”.

Database of State Tort Law Reforms (v.6.1)

Ronen Avraham (Tel Aviv/Texas) recently alerted me to an update of one of the more helpful resources around for those who study torts and tort reform. Specifically, the (DSTLR v.6.1) "updates the DSTLR (5th) and contains the most detailed, complete and comprehensive legal dataset of the most prevalent tort reforms in the United States between 1980 and 2018. The DSTLR has been downloaded more than 2700 times and has become the standard tool in empirical research of tort reform. The dataset records state laws in all fifty states and the District of Columbia over the last several decades...." For direct access to the dataset (in Excel) click here.

Power and More Power (Testing)

Power and sample-size calculations are an important part of planning a scientific study. In addition, too often papers that report (perhaps unexpected) null results do not take the next step and assess the possibility that any null results may be ascribed to an underpowered sample. While simple power tests are understood by many (click here for an inventory; click here for a helpful general intro video), power testing for more complex empirical specifications are not. A recent entry (the first of a series) on the Stata Blog (here) walks readers through power testing for more sophisticated modeling.

Marginal Effects Primer

Prof. Richard Williams (Notre Dame) put together an exceedingly helpful "primer" on Stata's "margins" command. Prof. WIlliam's PowerPoint slide deck (here) walks readers through the basics, provides helpful explanations, and illustrates though an array of concrete examples. As marginal effects gain in popularity many might find this helpful.

UPDATE: A loyal Els Blog reader, Prof. Jeff Yates (Binghamton), alerted me to two addition helpful links germane to margins (here and here).

Baldy Center Fellowships

The Baldy Center, an endowed, internationally recognized institute that advances interdisciplinary research on law, legal institutions, and social policy at the State University of New York at Buffalo, plans to appoint one new Postdoctoral Fellow and one or two new Senior Fellows for terms beginning during the 2019-20 academic year. These opportunities target scholars pursuing “important topics in law, legal institutions, and social policy.” Applications are invited from junior and senior scholars from law, the humanities, and the social and natural sciences.

Those with any questions should contact the Baldy Assistant Director Laura Wirth (baldyassistantdirector@gmail.com) directly. Please note that applications are due Wednesday, February 1, 2019.

Assessing Competing Criminal Defense Models

Recently-enacted criminal justice reform legislation reflects persistent public attention to nagging criminal law issues (it will also generate the variation that invites future empirical research). Another debate within the criminal law community pits "holistic" versus "traditional" criminal defense, both with competing empirical assumptions regarding efficacy. In a recent paper, The Effects of Holistic Defense on Criminal Justice Outcomes, James Anderson (RAND), et al. report results from a fascinating study.

The data set, while limited to Bronx County (NY), is significant. The paper exploits "an extract of all records from the DCJS Computerized Criminal History (CCH) database involving individuals arraigned within Bronx County between 2000 and the present, and supplemented these data with Bronx arraignment data from OCA. The CCH database includes all fingerprintable arrests that occur within the state of New York, and is the data source used to generate rap sheets following arrest for arraignments in the state. DCJS provided data covering over 2.8 million individual arrests involving nearly 400,000 distinct individuals." "The unit of observation is a defendant/case pairing, and we initially restrict attention to cases where arraignments occurred between 2000 and 2014 that had an initial disposition by October 2016, leaving a pool of 940,546 observations. We exclude the roughly 5% of Bronx criminal defendants who were not indigent and were represented by hired private counsel, as well as the as well as the <1% of cases where the DCJS file had missing data on counsel type.”

Complementing the data set is a clever research design that exploits a quasi-natural experiment and facilitates a helpful "holistic" versus "traditional" criminal defense comparison. “In the first half of the 2000s, the Bronx Defender shift assignments were centered on Mondays and Tuesdays, with the holistic defender covering all Monday and Tuesday shifts by 2004. In 2005, the Bronx Defenders began taking Sunday shifts, and they were temporarily moved to Thursday and Friday shifts at the end of 2005 before reverting to the prior arrangement in 2006. Beginning in 2012, the Defenders added Wednesday and Thursday shifts and began covering all Sunday and Monday shifts…. This rotating shift assignment pattern permits identification of the effects of holistic representation even when we control for day of week effects to account for the likelihood that crimes committed on particular days of the week, such as weekends, can be qualitatively different from those committed during the week.”

Key findings are described in the excerpted abstract: “This Article offers the first large-scale, rigorous evaluation of the impact of holistic representation on criminal justice outcomes. In the Bronx, a holistic defense provider (the Bronx Defenders) and a traditional defender (the Legal Aid Society) operate side-by-side within the same court system, with case assignment determined quasi-randomly based on court shift timing. Using administrative data covering over half a million cases and a quasi-experimental research design, we estimate the causal effect of holistic representation on case outcomes and future offending. Holistic representation does not affect conviction rates, but it reduces the likelihood of a custodial sentence by 16% and expected sentence length by 24%. Over the ten-year study period, holistic representation in the Bronx resulted in nearly 1.1 million fewer days of custodial punishment….”

Manipulating Date/Time Variables

An often needed--yet difficult to create--variable is a "duration" variable that seeks to capture time (or number of days) and is derived by "subtracting," e.g., a start-date from an end-date. This computation's degree of difficulty is enhanced by complexities incident to how dates (and time) are formatted in Stata. Those interested can find a recent step-by-step explanation (and illustration) here.

Empirical Legal Scholarship--An Empirical View From Europe

While (appropriately enough) empirical research on scholarship published in US-based law journals generally comports with the conventional wisdom regarding ELS' rise over time, results from a similar study of European-based law journals, however, point in a different direction. In Empirical Legal Research in Europe: Prevalence, Obstacles and Interventions, by van Dijck et al. (Maastricht, NL), the authors note while:

"There seems to be a strong belief that Empirical Legal Research (ELR) has been on the rise. A recently published article explored its prevalence in Europe. The authors carried out an empirical analysis of 78 European-based law journals, encompassing issues from 2008 to 2017. The findings demonstrate that a supposed increase of ELR is questionable (at best). Moreover, additional findings highlight prestigious journals being more likely to publish empirical articles than less-prestigious journals and older journals being more likely to publish empirical work than more recent journals, but not at an increasing rate. The study obviously comes with some limitations, since the time period that was examined is limited, because an analysis of the submitted articles may paint a different picture, or because ELR scholars may tend to publish in US-based journals rather than European-based journals. Nonetheless, the findings do raise the question why ELR has not become more popular in Europe."

The paper's selection-bias point is especially interesting (as is the nomenclature move from "ELS" to "ELR") and comports with JELS' editors sense of growing interest (evidenced by increased submissions) from non-US-based scholars.

Why Are "Happy" Women Lawyers Departing Law Practice? The Enduring Importance of Selection Effects

While women attorneys continue to depart the practice of law at persistently high rates, most survey research fails to detect evidence of gender differences in job satisfaction among attorneys. To explore this seeming paradox, Joni Hersch (Vanderbilt) and Erin Meyers (Vanderbilt--student) exploit data from the 2015 cohort of the National Survey of College Graduates (NSCG). The NSCG data set "contains 91,000 observations, with 1,928 having earned a JD. Our analysis sample includes respondents with a JD who are not retired and do not also have a PhD or MD. The resulting number of observations is 1,770. The NSCG provides detailed data on demographic characteristics, job characteristics, and job satisfaction, including a direct measure of overall job satisfaction."

Their paper, Why are Seemingly Satisfied Female Lawyers Running for the Exits? Resolving the Paradox Using National Data, argues that this "paradox" flows from selection effects. Specifically, the authors argue that the "apparent similarity of job satisfaction between genders likely arises from dissatisfied female JDs sorting out of the legal profession at higher rates than their male counterparts, leaving behind the most satisfied women." An excerpted abstract follows.

"This Article also provides a detailed examination of the specific working conditions that are associated with dissatisfaction for female lawyers before this sorting occurs, and compares job satisfaction of lawyers to that of other professions. The resulting analysis finds that recently graduated female lawyers have lower average satisfaction with their salaries relative to male lawyers, but that this result is likely due to differences in employer types. The data further indicates that a male-female satisfaction gap exists only among JDs, and not among those with other professional or graduate degrees. This finding suggests that analysis comparing lawyers to other professions could help to pinpoint unique characteristics of the legal profession that create dissatisfaction among women at the outset of their careers and causes them to exit the profession."

The Visual Display of Data--"Graphs v. Tables"

This enduring "chestnut" of an issue warrants periodic refreshing. An on-going discussion is found here. Those interested in the long-standing “classic” treatment of such issues, Edward Tufte’s, The Visual Display of Quantitative Information (2^nd ed., 2001) delves into the “theory and practice in the design of data graphics,” and includes illustrated examples of the “best (and a few of the worst) statistical graphics, with detailed analysis of how to display data for precise, effective, quick analysis.” Finally, in a 2011 essay Andrew Gelman (Columbia--Statistics) wades into these matters with a normative (and intentionally provocative) take favoring "tables over graphs." The Gelman paper's abstract follows.

"The statistical community is divided when it comes to graphical methods and models. Graphics researchers tend to disparage models and to focus on direct representations of data, mediated perhaps by research on perceptions but certainly not by probability distributions. From the other side, modelers tend to think of graphics as a cute toy for exploring raw data but not much help when it comes to the serious business of modeling. In order to better understand the benefits and limitations of graphs in statistical analysis, this article presents a series of criticisms of graphical methods in the voice of a hypothetical old-school analytical statistician or social scientist. We hope to elicit elaborations and extensions of these and other arguments on the limitations of graphics, along with responses from graphical researchers who might have different perceptions of these issues."

2019 CELS--Date and Location

On the heels of a stunningly successful 2018 CELS, hosted by the University of Michigan Law School and co-organized by Professors James R. Hines Jr., J.J. Prescott, and Sonja B. Starr, I wanted to alert those interested that the 2019 CELS, scheduled for November 8-9, 2019, will be hosted by Claremont McKenna College (Calif.) and co-organized by Professors David Bjerk, Eric Helland, and Dan Krauss.

Data & Property Rights

While the array of issues incident to "data" and property rights (or "ownership") continues to grow both in number and complexity, many of the immediate, practical questions relate to the evolving relation between empirical scholars and their data sets and journal editors when it comes to the manuscript review process. A wider-angle lens, however, as well as an international perspective, are developed in a recent, brief working paper/report, Data Ownership. In it, Teresa Scassa (Ottawa) takes up broader questions concerning data "ownership," what ownership may mean, and how whatever it may mean can vary across jurisdictions. While certainly not the last word on these complex issues, an interesting take and perspective on a set of issues that will not likely go away anytime soon.

Extended Submission Deadline--Program on Empirical Legal Studies (PELS)--2019 Replication Conference

Extended Submission Deadline: Friday, Nov. 16, 2018, midnight (Pacific time).

I'm delighted to announce that Claremont McKenna College's Program on Empirical Legal Studies (PELS) is accepting paper proposals for its Second Annual Replication Conference, scheduled for Friday, April 26, 2019, in Claremont, CA.

The call for paper proposals notes: "We are currently soliciting proposals that detail a plan to replicate an important study in law and economics/empirical legal studies. We welcome submissions using all methodologies and the conference will include a range of replication methodologies. We encourage proposals from teams consisting of a faculty member and one or more graduate students, in recognition of the fact that replication studies are also an excellent form of pedagogy."

For those interested click here for additional information.

"Driving While White" (During Stressful Budgetary Times)

Much of late has been made of: 1) "problematic" policing in general and 2) police activity that appears to be "budget-driven." A recent paper by Elliot Ash (Warwick) et al., Local Public Finance and Discriminatory Policing: Evidence from Traffic Stops in Missouri, takes an interesting step at trying to connect some of these dots.

As to "problematic" policing practices, the paper references a "large literature has documented that traffic police disproportionately target black and Hispanic drivers when making stops." Another seemingly unrelated literature explores the "public-finance motivations underlying aggressive policing. It is well-documented that local governments rely on revenue from traffic tickets, and officials often look to this source of revenue to help overcome budget shortfalls."

This paper sets out to empirically explore 'the intersection of race, policing, and this form of latent taxation, to determine whether the burden to close budget gaps falls disproportionately on non-white motorists and pedestrians." Specifically, the authors assess whether "police officers’ ticketing behaviors are discriminatory, and in particular, whether the disparity in ticketing changes when a municipality is faced with governmental pressures to increase ticketing revenue."

To do so the paper exploits two main sources of data. One source involves Missouri government finance data. A second source involves "agency-level traffic stops data, used to construct measures of traffic enforcement effort across races. There are 769 agencies in the sample, for which we have 13 years of annual panel data from 2000 through 2012."

While part of the paper's findings will unlikely surprise many, another part likely will. First, as it relates to "public-finance [or budget] motivated police conduct," the authors find that a "decrease in government revenue growth the previous year is associated with a higher citation rate." Second, as it relates to discriminatory policing, the authors find that this budget-stress-motivated increased enforcement activity (tickets and arrests) fell disproportionately on white drivers and not on non-white drivers. The paper's abstract follows.

"This paper provides evidence of racial variation in local governments’ traffic enforcement responses to budget stress using data from policing agencies in the state of Missouri for the years 2001 through 2014. Like previous studies, we find that local budget stress is associated with higher citation rates. In addition, we find that there is an increase in traffic-stop arrests. However, we find that these effects are concentrated among white (rather than black or Hispanic) drivers. This statistical difference is robust to the inclusion of a range of covariates for traffic stops and to the inclusion of local population features interacted with year. These results are consistent with a model where traffic police selectively target higher-income drivers to compensate for budget stress. Also consistent with this view, we find that the racial difference in citation and arrest rates is highest where the white-to-black income ratio is highest."