Empirical Legal Studies

Manipulating Date/Time Variables

An often needed--yet difficult to create--variable is a "duration" variable that seeks to capture time (or number of days) and is derived by "subtracting," e.g., a start-date from an end-date. This computation's degree of difficulty is enhanced by complexities incident to how dates (and time) are formatted in Stata. Those interested can find a recent step-by-step explanation (and illustration) here.

Empirical Legal Scholarship--An Empirical View From Europe

While (appropriately enough) empirical research on scholarship published in US-based law journals generally comports with the conventional wisdom regarding ELS' rise over time, results from a similar study of European-based law journals, however, point in a different direction. In Empirical Legal Research in Europe: Prevalence, Obstacles and Interventions, by van Dijck et al. (Maastricht, NL), the authors note while:

"There seems to be a strong belief that Empirical Legal Research (ELR) has been on the rise. A recently published article explored its prevalence in Europe. The authors carried out an empirical analysis of 78 European-based law journals, encompassing issues from 2008 to 2017. The findings demonstrate that a supposed increase of ELR is questionable (at best). Moreover, additional findings highlight prestigious journals being more likely to publish empirical articles than less-prestigious journals and older journals being more likely to publish empirical work than more recent journals, but not at an increasing rate. The study obviously comes with some limitations, since the time period that was examined is limited, because an analysis of the submitted articles may paint a different picture, or because ELR scholars may tend to publish in US-based journals rather than European-based journals. Nonetheless, the findings do raise the question why ELR has not become more popular in Europe."

The paper's selection-bias point is especially interesting (as is the nomenclature move from "ELS" to "ELR") and comports with JELS' editors sense of growing interest (evidenced by increased submissions) from non-US-based scholars.

Why Are "Happy" Women Lawyers Departing Law Practice? The Enduring Importance of Selection Effects

While women attorneys continue to depart the practice of law at persistently high rates, most survey research fails to detect evidence of gender differences in job satisfaction among attorneys. To explore this seeming paradox, Joni Hersch (Vanderbilt) and Erin Meyers (Vanderbilt--student) exploit data from the 2015 cohort of the National Survey of College Graduates (NSCG). The NSCG data set "contains 91,000 observations, with 1,928 having earned a JD. Our analysis sample includes respondents with a JD who are not retired and do not also have a PhD or MD. The resulting number of observations is 1,770. The NSCG provides detailed data on demographic characteristics, job characteristics, and job satisfaction, including a direct measure of overall job satisfaction."

Their paper, Why are Seemingly Satisfied Female Lawyers Running for the Exits? Resolving the Paradox Using National Data, argues that this "paradox" flows from selection effects. Specifically, the authors argue that the "apparent similarity of job satisfaction between genders likely arises from dissatisfied female JDs sorting out of the legal profession at higher rates than their male counterparts, leaving behind the most satisfied women." An excerpted abstract follows.

"This Article also provides a detailed examination of the specific working conditions that are associated with dissatisfaction for female lawyers before this sorting occurs, and compares job satisfaction of lawyers to that of other professions. The resulting analysis finds that recently graduated female lawyers have lower average satisfaction with their salaries relative to male lawyers, but that this result is likely due to differences in employer types. The data further indicates that a male-female satisfaction gap exists only among JDs, and not among those with other professional or graduate degrees. This finding suggests that analysis comparing lawyers to other professions could help to pinpoint unique characteristics of the legal profession that create dissatisfaction among women at the outset of their careers and causes them to exit the profession."

The Visual Display of Data--"Graphs v. Tables"

This enduring "chestnut" of an issue warrants periodic refreshing. An on-going discussion is found here. Those interested in the long-standing “classic” treatment of such issues, Edward Tufte’s, The Visual Display of Quantitative Information (2^nd ed., 2001) delves into the “theory and practice in the design of data graphics,” and includes illustrated examples of the “best (and a few of the worst) statistical graphics, with detailed analysis of how to display data for precise, effective, quick analysis.” Finally, in a 2011 essay Andrew Gelman (Columbia--Statistics) wades into these matters with a normative (and intentionally provocative) take favoring "tables over graphs." The Gelman paper's abstract follows.

"The statistical community is divided when it comes to graphical methods and models. Graphics researchers tend to disparage models and to focus on direct representations of data, mediated perhaps by research on perceptions but certainly not by probability distributions. From the other side, modelers tend to think of graphics as a cute toy for exploring raw data but not much help when it comes to the serious business of modeling. In order to better understand the benefits and limitations of graphs in statistical analysis, this article presents a series of criticisms of graphical methods in the voice of a hypothetical old-school analytical statistician or social scientist. We hope to elicit elaborations and extensions of these and other arguments on the limitations of graphics, along with responses from graphical researchers who might have different perceptions of these issues."

2019 CELS--Date and Location

On the heels of a stunningly successful 2018 CELS, hosted by the University of Michigan Law School and co-organized by Professors James R. Hines Jr., J.J. Prescott, and Sonja B. Starr, I wanted to alert those interested that the 2019 CELS, scheduled for November 8-9, 2019, will be hosted by Claremont McKenna College (Calif.) and co-organized by Professors David Bjerk, Eric Helland, and Dan Krauss.

Data & Property Rights

While the array of issues incident to "data" and property rights (or "ownership") continues to grow both in number and complexity, many of the immediate, practical questions relate to the evolving relation between empirical scholars and their data sets and journal editors when it comes to the manuscript review process. A wider-angle lens, however, as well as an international perspective, are developed in a recent, brief working paper/report, Data Ownership. In it, Teresa Scassa (Ottawa) takes up broader questions concerning data "ownership," what ownership may mean, and how whatever it may mean can vary across jurisdictions. While certainly not the last word on these complex issues, an interesting take and perspective on a set of issues that will not likely go away anytime soon.

Extended Submission Deadline--Program on Empirical Legal Studies (PELS)--2019 Replication Conference

Extended Submission Deadline: Friday, Nov. 16, 2018, midnight (Pacific time).

I'm delighted to announce that Claremont McKenna College's Program on Empirical Legal Studies (PELS) is accepting paper proposals for its Second Annual Replication Conference, scheduled for Friday, April 26, 2019, in Claremont, CA.

The call for paper proposals notes: "We are currently soliciting proposals that detail a plan to replicate an important study in law and economics/empirical legal studies. We welcome submissions using all methodologies and the conference will include a range of replication methodologies. We encourage proposals from teams consisting of a faculty member and one or more graduate students, in recognition of the fact that replication studies are also an excellent form of pedagogy."

For those interested click here for additional information.

"Driving While White" (During Stressful Budgetary Times)

Much of late has been made of: 1) "problematic" policing in general and 2) police activity that appears to be "budget-driven." A recent paper by Elliot Ash (Warwick) et al., Local Public Finance and Discriminatory Policing: Evidence from Traffic Stops in Missouri, takes an interesting step at trying to connect some of these dots.

As to "problematic" policing practices, the paper references a "large literature has documented that traffic police disproportionately target black and Hispanic drivers when making stops." Another seemingly unrelated literature explores the "public-finance motivations underlying aggressive policing. It is well-documented that local governments rely on revenue from traffic tickets, and officials often look to this source of revenue to help overcome budget shortfalls."

This paper sets out to empirically explore 'the intersection of race, policing, and this form of latent taxation, to determine whether the burden to close budget gaps falls disproportionately on non-white motorists and pedestrians." Specifically, the authors assess whether "police officers’ ticketing behaviors are discriminatory, and in particular, whether the disparity in ticketing changes when a municipality is faced with governmental pressures to increase ticketing revenue."

To do so the paper exploits two main sources of data. One source involves Missouri government finance data. A second source involves "agency-level traffic stops data, used to construct measures of traffic enforcement effort across races. There are 769 agencies in the sample, for which we have 13 years of annual panel data from 2000 through 2012."

While part of the paper's findings will unlikely surprise many, another part likely will. First, as it relates to "public-finance [or budget] motivated police conduct," the authors find that a "decrease in government revenue growth the previous year is associated with a higher citation rate." Second, as it relates to discriminatory policing, the authors find that this budget-stress-motivated increased enforcement activity (tickets and arrests) fell disproportionately on white drivers and not on non-white drivers. The paper's abstract follows.

"This paper provides evidence of racial variation in local governments’ traffic enforcement responses to budget stress using data from policing agencies in the state of Missouri for the years 2001 through 2014. Like previous studies, we find that local budget stress is associated with higher citation rates. In addition, we find that there is an increase in traffic-stop arrests. However, we find that these effects are concentrated among white (rather than black or Hispanic) drivers. This statistical difference is robust to the inclusion of a range of covariates for traffic stops and to the inclusion of local population features interacted with year. These results are consistent with a model where traffic police selectively target higher-income drivers to compensate for budget stress. Also consistent with this view, we find that the racial difference in citation and arrest rates is highest where the white-to-black income ratio is highest."

Longitudinal Studies

When appropriate to a research question(s) and where resources permit, longitudinal studies can provide important insights that are not possible from studies that use the more common cross-sectional approach. In longitudinal studies, researchers engage the same participants with repeated measures over time. While perhaps somewhat unusual in the legal research literatures, longitudinal studies are not unknown.

A recent paper by John Monahan (UVa) and Jeff Swanson (Duke--psych), Lawyers at the Peak of Their Careers: A 30-Year Longitudinal Study of Job and Life Satisfaction, provides one such example. The study follows a cohort of UVa law students who matriculated in 1987. Critical to the success of longitudinal studies, of course, involves censoring attributable to sample participation decay over time. In their original paper the researchers (in 2007) engaged with 58% of those law students who entered UVa law school 20 years earlier in 1987. Their recent follow-up paper, reporting results from their 2017 follow-up survey, includes 81% of those who participated one decade earlier (or, to be more precise, the 2017 sample includes 81% of the 58% who participated ten years prior in 2007). A summary of key findings, drawn from the abstract, follows.

“We found respondents to have taken diverse career paths, with no single work setting accounting for more than one-quarter of the respondents and with fully one-third of the respondents changing jobs in the past decade. Marked gender differences in the professional lives of respondents persisted (e.g., women continued to be much more likely than men to forego full-time employment 'in order to care for children' (30 percent vs. 4 percent)). Working conditions at large private law firms stayed problematic, with the portion of respondents negatively affected by a stronger stress on economic sustainability being twice as high among those working in large firms (77 percent) than among those working in other settings (38 percent). Finally, both career satisfaction and life satisfaction again were found to be high, with 77 percent of respondents satisfied with their decision to become a lawyer, and 91 percent satisfied with their lives more broadly.”

The Effects of Miranda on Crime Clearance Rates

Paul Cassell (Utah), who interrupted his academic career to serve as a federal district court judge (2002-07), has been a long-term critic of the 1966 Miranda decision. In a recent paper co-authored with Richard Fowles (Utah--econ), Still Handcuffing the Cops After all These Years: A Review of Fifty Years of Empirical Evidence of Miranda’s Harmful Effects on Law Enforcement, Cassell and Fowles extend their critique from the theoretical and into the empirical realm. Results from their study suggest that the Miranda decision contributed to a statistically significant reduction in crime clearance rates between 1950-2012 for an array of crime types. The abstract follows.

"The fiftieth anniversary of Miranda v. Arizona offers a chance to assess how the decision has played out in the real world and, in particular, to determine whether it has harmed law enforcement. In this Article, we take advantage of the time since the Miranda decision—now a little more than fifty years—to see whether it has produced the predicted harmful consequences. In particular, we survey the available empirical evidence about Miranda’s effects on law enforcement. We collect confession rate data, both from the time of Miranda and since, to assess whether Miranda caused confession rates to fall. We also review the FBI’s nationwide data on crime clearance rates to shed light on any changes in the ability of police to solve crimes. Specifically, we report the results of regression equations on crime clearance rates from 1950 to 2012, controlling for factors apart from Miranda that might be responsible for changes in clearance rates. Even controlling for these factors, we find statistically significant reductions in crime clearance rates after Miranda for violent and property crimes, as well as for robbery, larceny, and vehicle theft. We also quantify the number of lost clearances that appear to be due to Miranda. We also briefly conclude by encouraging the Supreme Court, as well as commentators and policy makers, to consider alternative ways of regulating police interrogation that do not have such detrimental effects on police efforts to apprehend potentially dangerous criminals."

Research Design and Multi-Level Modeling--An Illustration

A recent post on Andrew Gelman's blog (here)--ironically prompted by a grad student's frustration with efforts to detect treatment effects in, of all things, the weight-training context--caught my eye not for any insights into weight-training but, rather, for some of the post's practical research design and multi-level modeling suggestions.

An truncated version of the student's question follows: "Consider the field of exercise science, and in particular studies on people who lift weights... Because muscle gain, fat loss, and (to a lesser extent) strength gain happen very slowly, and these studies usually last a few weeks to a few months at most, the effect sizes are all quite small. This is especially the case when comparing any two interventions that are not wholly idiotic for a difference in means."

Gelman's multi-step reply begins with the assumption that "the way to go is to have detailed within-person trajectories. Crude between-person designs just won’t cut it...." Executing a within-person trajectory approach from a research design and data-gathering perspectives involves: "At the design stage, it would be best to try multiple treatments per person. And you’ll want multiple, precise measurements. This last point sounds kinda obvious, but we don’t always see it because researchers have been able to achieve apparent success through statistical significance with any data at all." Gelman's final suggestion focuses on model selection. "The next question is how to analyze the data. You’ll want to fit a multilevel model with a different trajectory for each person: most simply, a varying-intercept, varying-slope model."

While the example in the post features R code, it nicely illustrates some of the critical ex ante decisions necessary for successful empirical studies.

Civil Trials as "Dodo" Birds: From "Vanishing" to "Diminished"

Empirical work, including important scholarship by, among others, Marc Galanter (Wisconsin), has persuasively documented a steady decline of civil litigation concluded by formal trial. And this "vanishing trial" trend straddles federal and states courts. In a recent paper, The Diminished Trial, Nora Freeman Engstrom (Stanford) nudges this research a bit further by taking an empirical dive into what the pool of remaining civil trials look like. Engstrom isolates two key trends: the steep decline in "protracted" trials (trials lasting 20 or more days) and the concurrent sharp rise in "really short" trials (those lasting one day or less). And the interaction of these two trends prompts Engstrom to characterize civil litigation today as "diminished. The paper's abstract follows.

“Civil trials, many have noted, are going the way of the dodo bird. Federal courts conducted half as many civil trials in 2016 as they did in 1962, even while disposing of over five times as many civil cases. A similar trend is apparent in the states. Of course, this trajectory has not escaped scholarly attention. Barrels of ink have been spilled investigating, eulogizing, and variously, mourning or lauding, the “vanishing trial.” But, this is far from the whole story. This Article shifts the conversation to a different, though related, phenomenon: not the disappearance of the civil trial, but rather, its downsizing. I take as my point of departure two puzzling trends. First, since 1983, “protracted” trials, which is to say trials that last over 20 days, are way down. There were more than 100 such trials per year in the late 1980s, but in 2016, we only saw only 13. Second, over the same timespan, really short trials, which is to say trials lasting one day or less, are up. Indeed, starting in 2007, and every year since, the majority of all federal civil trials have wrapped up in only one day. This paper seeks to highlight these trends, which have so far escaped scholarly attention, and also to conduct a preliminary investigation into the potential causes and consequences.”

"Checking the Basics": Regression Assumptions

Too often researchers neglect the basics that they learned in grad school (or, perhaps, as undergrads) regarding regression assumptions. A recent Statalist discussion (here) highlights some of the basics when it comes to standard OLS regression "diagnostics." The commentary also underscores that there is no single "correct" approach and that level-headed judgment is at a premium. Finally, as an early comment notes, searching on "regress postestimation" and "regress postestimation plots" in Stata's "Help" tab uncovers a treasure-trove of helpful commands.

The Topography of Civil Litigation and Plaintiff Attorneys' Role

In Litigation Migrants, Charlotte Alexander (Georgia State) draws on data from FLSA litigation to help develop a theory of "litigation migration." Seeking to explain trends and changes in FLSA litigation over time (and its geographic movement over time) and exploiting a sample of FLSA cases from 2000-2016, the author finds that "personal injury lawyers have migrated across case types to take on FLSA cases, that lawyers have migrated across geographic boundaries to take on FLSA cases in other jurisdictions, and that both types of migration have influenced FLSA case filing numbers.” What the author argues that the "FLSA boom provides a remarkable opportunity to study what John Coffee has called 'entrepreneurial litigation,' in full flower. Indeed, the litigation migrants studied here have remade the wage and hour landscape, recovering many millions of dollars for workers and adding to the list of worries that cause employers to lose sleep." From the data the paper forward's an attorney-centric account. " Paraphrasing Hyman and Silver (“It’s the [plaintiffs’ lawyers’] incentives, stupid,”), Alexander's account emphasizes plaintiffs’ lawyers as a key litigation driver. Accordingly, "statutory or regulatory [efforts], such as tort reform, to shut down those lawyers’ income streams may result in a caseload shift, and a flight to more remunerative specialties. Admittedly, the tort reform variable in the two-part regression model described above produced no statistically significant results. However, because data availability limited the study period to 2000-2016, the model may have failed to capture the true impact of tort reform that occurred earlier, such as Florida’s set of reforms enacted in 1999.” The paper's abstract follows.

"Civil law is enforced primarily via private litigation. One characteristic of private enforcement is that litigation levels tend to cycle between booms and busts. This Article builds a theory for explaining this fluctuation, proposing that plaintiffs' attorneys can be understood as economic migrants. Just as workers cross borders to find jobs, lawyers 'move' across case types and jurisdictions to find profitable claims, and case-filing numbers increase as a result. Using the recent wage and hour litigation boom as a case study, this Article paints an empirical picture of attorney migration and its influence on case filing numbers. Drawing on these analyses, the Article concludes by considering the optimal litigation level within our private enforcement system.

Importing (and "Destring"-ing) Data From Excel (Again)

Practical questions on how to import Excel data sets into Stata seem to follow a five-year (or so) cycle. While invaluable (and relatively inexpensive) data conversion software (e.g., StatTransfer) can mange most situations, how Stata "reads" imported Excel data that may include a blend of "alpha" and "numeric" data can trip folks up. A recent discussion on the Stata Listserv (here) on the utility of Stata's "destring" command may assist.