Empirical Legal Studies

Longitudinal Studies

When appropriate to a research question(s) and where resources permit, longitudinal studies can provide important insights that are not possible from studies that use the more common cross-sectional approach. In longitudinal studies, researchers engage the same participants with repeated measures over time. While perhaps somewhat unusual in the legal research literatures, longitudinal studies are not unknown.

A recent paper by John Monahan (UVa) and Jeff Swanson (Duke--psych), Lawyers at the Peak of Their Careers: A 30-Year Longitudinal Study of Job and Life Satisfaction, provides one such example. The study follows a cohort of UVa law students who matriculated in 1987. Critical to the success of longitudinal studies, of course, involves censoring attributable to sample participation decay over time. In their original paper the researchers (in 2007) engaged with 58% of those law students who entered UVa law school 20 years earlier in 1987. Their recent follow-up paper, reporting results from their 2017 follow-up survey, includes 81% of those who participated one decade earlier (or, to be more precise, the 2017 sample includes 81% of the 58% who participated ten years prior in 2007). A summary of key findings, drawn from the abstract, follows.

“We found respondents to have taken diverse career paths, with no single work setting accounting for more than one-quarter of the respondents and with fully one-third of the respondents changing jobs in the past decade. Marked gender differences in the professional lives of respondents persisted (e.g., women continued to be much more likely than men to forego full-time employment 'in order to care for children' (30 percent vs. 4 percent)). Working conditions at large private law firms stayed problematic, with the portion of respondents negatively affected by a stronger stress on economic sustainability being twice as high among those working in large firms (77 percent) than among those working in other settings (38 percent). Finally, both career satisfaction and life satisfaction again were found to be high, with 77 percent of respondents satisfied with their decision to become a lawyer, and 91 percent satisfied with their lives more broadly.”

The Effects of Miranda on Crime Clearance Rates

Paul Cassell (Utah), who interrupted his academic career to serve as a federal district court judge (2002-07), has been a long-term critic of the 1966 Miranda decision. In a recent paper co-authored with Richard Fowles (Utah--econ), Still Handcuffing the Cops After all These Years: A Review of Fifty Years of Empirical Evidence of Miranda’s Harmful Effects on Law Enforcement, Cassell and Fowles extend their critique from the theoretical and into the empirical realm. Results from their study suggest that the Miranda decision contributed to a statistically significant reduction in crime clearance rates between 1950-2012 for an array of crime types. The abstract follows.

"The fiftieth anniversary of Miranda v. Arizona offers a chance to assess how the decision has played out in the real world and, in particular, to determine whether it has harmed law enforcement. In this Article, we take advantage of the time since the Miranda decision—now a little more than fifty years—to see whether it has produced the predicted harmful consequences. In particular, we survey the available empirical evidence about Miranda’s effects on law enforcement. We collect confession rate data, both from the time of Miranda and since, to assess whether Miranda caused confession rates to fall. We also review the FBI’s nationwide data on crime clearance rates to shed light on any changes in the ability of police to solve crimes. Specifically, we report the results of regression equations on crime clearance rates from 1950 to 2012, controlling for factors apart from Miranda that might be responsible for changes in clearance rates. Even controlling for these factors, we find statistically significant reductions in crime clearance rates after Miranda for violent and property crimes, as well as for robbery, larceny, and vehicle theft. We also quantify the number of lost clearances that appear to be due to Miranda. We also briefly conclude by encouraging the Supreme Court, as well as commentators and policy makers, to consider alternative ways of regulating police interrogation that do not have such detrimental effects on police efforts to apprehend potentially dangerous criminals."

Research Design and Multi-Level Modeling--An Illustration

A recent post on Andrew Gelman's blog (here)--ironically prompted by a grad student's frustration with efforts to detect treatment effects in, of all things, the weight-training context--caught my eye not for any insights into weight-training but, rather, for some of the post's practical research design and multi-level modeling suggestions.

An truncated version of the student's question follows: "Consider the field of exercise science, and in particular studies on people who lift weights... Because muscle gain, fat loss, and (to a lesser extent) strength gain happen very slowly, and these studies usually last a few weeks to a few months at most, the effect sizes are all quite small. This is especially the case when comparing any two interventions that are not wholly idiotic for a difference in means."

Gelman's multi-step reply begins with the assumption that "the way to go is to have detailed within-person trajectories. Crude between-person designs just won’t cut it...." Executing a within-person trajectory approach from a research design and data-gathering perspectives involves: "At the design stage, it would be best to try multiple treatments per person. And you’ll want multiple, precise measurements. This last point sounds kinda obvious, but we don’t always see it because researchers have been able to achieve apparent success through statistical significance with any data at all." Gelman's final suggestion focuses on model selection. "The next question is how to analyze the data. You’ll want to fit a multilevel model with a different trajectory for each person: most simply, a varying-intercept, varying-slope model."

While the example in the post features R code, it nicely illustrates some of the critical ex ante decisions necessary for successful empirical studies.

Civil Trials as "Dodo" Birds: From "Vanishing" to "Diminished"

Empirical work, including important scholarship by, among others, Marc Galanter (Wisconsin), has persuasively documented a steady decline of civil litigation concluded by formal trial. And this "vanishing trial" trend straddles federal and states courts. In a recent paper, The Diminished Trial, Nora Freeman Engstrom (Stanford) nudges this research a bit further by taking an empirical dive into what the pool of remaining civil trials look like. Engstrom isolates two key trends: the steep decline in "protracted" trials (trials lasting 20 or more days) and the concurrent sharp rise in "really short" trials (those lasting one day or less). And the interaction of these two trends prompts Engstrom to characterize civil litigation today as "diminished. The paper's abstract follows.

“Civil trials, many have noted, are going the way of the dodo bird. Federal courts conducted half as many civil trials in 2016 as they did in 1962, even while disposing of over five times as many civil cases. A similar trend is apparent in the states. Of course, this trajectory has not escaped scholarly attention. Barrels of ink have been spilled investigating, eulogizing, and variously, mourning or lauding, the “vanishing trial.” But, this is far from the whole story. This Article shifts the conversation to a different, though related, phenomenon: not the disappearance of the civil trial, but rather, its downsizing. I take as my point of departure two puzzling trends. First, since 1983, “protracted” trials, which is to say trials that last over 20 days, are way down. There were more than 100 such trials per year in the late 1980s, but in 2016, we only saw only 13. Second, over the same timespan, really short trials, which is to say trials lasting one day or less, are up. Indeed, starting in 2007, and every year since, the majority of all federal civil trials have wrapped up in only one day. This paper seeks to highlight these trends, which have so far escaped scholarly attention, and also to conduct a preliminary investigation into the potential causes and consequences.”

"Checking the Basics": Regression Assumptions

Too often researchers neglect the basics that they learned in grad school (or, perhaps, as undergrads) regarding regression assumptions. A recent Statalist discussion (here) highlights some of the basics when it comes to standard OLS regression "diagnostics." The commentary also underscores that there is no single "correct" approach and that level-headed judgment is at a premium. Finally, as an early comment notes, searching on "regress postestimation" and "regress postestimation plots" in Stata's "Help" tab uncovers a treasure-trove of helpful commands.

The Topography of Civil Litigation and Plaintiff Attorneys' Role

In Litigation Migrants, Charlotte Alexander (Georgia State) draws on data from FLSA litigation to help develop a theory of "litigation migration." Seeking to explain trends and changes in FLSA litigation over time (and its geographic movement over time) and exploiting a sample of FLSA cases from 2000-2016, the author finds that "personal injury lawyers have migrated across case types to take on FLSA cases, that lawyers have migrated across geographic boundaries to take on FLSA cases in other jurisdictions, and that both types of migration have influenced FLSA case filing numbers.” What the author argues that the "FLSA boom provides a remarkable opportunity to study what John Coffee has called 'entrepreneurial litigation,' in full flower. Indeed, the litigation migrants studied here have remade the wage and hour landscape, recovering many millions of dollars for workers and adding to the list of worries that cause employers to lose sleep." From the data the paper forward's an attorney-centric account. " Paraphrasing Hyman and Silver (“It’s the [plaintiffs’ lawyers’] incentives, stupid,”), Alexander's account emphasizes plaintiffs’ lawyers as a key litigation driver. Accordingly, "statutory or regulatory [efforts], such as tort reform, to shut down those lawyers’ income streams may result in a caseload shift, and a flight to more remunerative specialties. Admittedly, the tort reform variable in the two-part regression model described above produced no statistically significant results. However, because data availability limited the study period to 2000-2016, the model may have failed to capture the true impact of tort reform that occurred earlier, such as Florida’s set of reforms enacted in 1999.” The paper's abstract follows.

"Civil law is enforced primarily via private litigation. One characteristic of private enforcement is that litigation levels tend to cycle between booms and busts. This Article builds a theory for explaining this fluctuation, proposing that plaintiffs' attorneys can be understood as economic migrants. Just as workers cross borders to find jobs, lawyers 'move' across case types and jurisdictions to find profitable claims, and case-filing numbers increase as a result. Using the recent wage and hour litigation boom as a case study, this Article paints an empirical picture of attorney migration and its influence on case filing numbers. Drawing on these analyses, the Article concludes by considering the optimal litigation level within our private enforcement system.

Importing (and "Destring"-ing) Data From Excel (Again)

Practical questions on how to import Excel data sets into Stata seem to follow a five-year (or so) cycle. While invaluable (and relatively inexpensive) data conversion software (e.g., StatTransfer) can mange most situations, how Stata "reads" imported Excel data that may include a blend of "alpha" and "numeric" data can trip folks up. A recent discussion on the Stata Listserv (here) on the utility of Stata's "destring" command may assist.

Online Opt-In Surveys, Sample Weights, and Accuracy

For some obvious and less-than-obvious reasons, polling in general and online opt-in polling in particular have come under increased scrutiny. Despite their increased popularity (owing partly to cost factors), a recent Pew Research Center Report notes that such surveys have "raised some concern within the industry because, while low participation rates pose a challenge for all surveys, the online opt-in variety face additional hurdles." In many instances researchers turn to various sample weighting adjustments in an effort to off-set sample biases. While the brief Report can speak for itself, one clear take-away is that: "Even the most effective adjustment procedures were unable to remove most of the bias." Interestingly, "when it comes to accuracy, choosing the right variables for weighting is more important than choosing the right statistical [adjustment or 'sample weighting'] method." Andrew Gelman's (Columbia--Statistics) comments on the report are found here.

Substantive v. Statistical Factors in Model Selection

In a recent Stata list discussion, a grad student investigating firms' leverage ratios raised a question about whether to run a fixed-effects or OLS model on some panel data and cluster. To assess his results' robustness to model specifications, he ran the data across an array of models (below).

The advice offered, however, emphasized that "the answer to your ... question comes from substantive finance considerations, not statistics or Stata." That is, in some instances the precise nature of your research question and context, in combination with research design and data structure factors, inform specification selection where various options are, from a purely statistical perspective, "not necessarily wrong."

D-I-D Slides

Compliments of Clyde Schechter (a prominent blogger on the Stata List) those interested in a very helpful set of basic difference-in-difference slides can click here. The slides summarize various D-I-D basic designs, key assumptions, threats to validity, and related matters.

2018 CLEA--Call For Papers

The Canadian Law & Economics Association (CLEA) will hold its annual conference at the University of Toronto Faculty of Law on September 21-22, 2018. Professor Robert E. Scott (Columbia) will deliver the Tory Law & Economics Public Lecture. The conference provides an opportunity for law and economic scholars to present new, original research in areas including artificial intelligence, bankruptcy, competition, corporate law, development, environmental, finance, judicial behavior, property, and tax. For those interested, more information on CLEA’s call for papers can be found here.

Please note that the submission deadline is July 31, 2018.

Death Penalty Administration in Black and White

Sophisticated empirical work on disparities with the administration of capital punishment began in earnest decades ago. In the mid-1980s Baldus et al. (and others) made important insights by delving into procedural disparities that precede prosecutorial charging decisions and jurors’ sentencing decisions. In the runup to the McClesky decision, Baldus and Woodworth documented that "a Black defendant accused of killing a White victim in Georgia was 3.1 times more likely than any defendant accused of killing a Black victim in the years immediately after Gregg."

A recent paper by Fagan (Columbia) and Geller (NYU-Sociology), Police, Race, and the Production of Capital Homicides, advances our understanding by taking a step backwards. Specifically, the authors argue that the "emphasis on prosecutorial decisions overlooks a critical stage in the production of death penalty cases: police investigations and arrests. Prosecutors select cases for capital prosecution from a pool of intentional homicides created predominantly through police investigations and arrests. To an extent previously unknown, disparities in charging may reflect antecedent racial biases in the production of capital-eligible homicides by the police. That production process is our focus.” An excerpted abstract follows.

“Racial disparities in capital punishment have been well documented for decades. Over 50 studies have shown that Black defendants more likely than their white counterparts to be charged with capital-eligible crimes, to be convicted and sentenced to death. Racial disparities in charging and sentencing in capital-eligible homicides are the largest for the small number of cases where black defendants murder white victims compared to within race killings, or where whites murder black or other ethnic minority victims. These patterns are robust to rich controls for non-racial characteristics and state sentencing guidelines. This article backs up the research on racial disparities to an earlier stage of capital case processing: the production of capital-eligible cases beginning with the identification of potential defendants by the police. It seeks to trace these sentencing disparities to examining earlier stages in the processing of homicides. Using data from the FBI’s Supplementary Homicide Reports, we examine every homicide reported between 1976 and 2009, and find that homicides with white victims are significantly more likely to be “cleared” by the arrest of a suspect than are homicides with minority victims. We estimate a series of hierarchical regressions to show that a substantial portion of this disparity is explained by social and demographic characteristics of the county in which homicides take place. Most notably, counties with large concentrations of minority residents have lower clearance rates than do predominantly white counties; however, county characteristics do not fully explain the observed race-of-victim disparities.”

Another Dedicated Empirical Legal Scholarship Law Review Issue: Virginia Environmental Law Journal (Spring 2019)

A colleague at UVa passed along the following call for papers (below) for the benefit of empirical environmental law scholars. (On a related aside, I'm beginning to detect a faint increase in the number of traditional student-edited law reviews moving in a direction that more closely resembles peer-reviewed journals.)

VELJ Empirical Issue Submissions

The Virginia Environmental Law Journal (VELJ) is calling for empirical articles, which will be published in its first annual empirical legal scholarship issue in Spring 2019. We are looking for pieces that discuss environmental law topics by utilizing quantitative, qualitative, and mixed methods.

The submission process begins June 15 and concludes September 15, 2018. Publication decisions will be released no later than December 1, 2018. The submission process is exclusive, such that authors who submit to VELJ may not submit to other publications until receiving a decision from the VELJ Editorial Board. If an offer is extended, the author must accept (i.e. the offer is binding).

The articles will be reviewed by a double-blind peer review process. Please submit a cover letter (or similar email), the article (in a .doc or .docx format), a CV, and any supporting materials (detailed below) to: velj.submissions@gmail.com

Length and Formatting

• VELJ strongly prefers articles ranging from 10,000 to 15,000 words (footnotes included).
• Manuscript should be double-spaced, utilize footnotes, and conform to The Bluebook: A Uniform System of Citation (19th ed. 2010).

Supporting Materials

• Abrief explanation of the study design, methodology, and/or analysis of the empirical work underlying the article.
• Substantiating data sets and/or supporting material are welcome at the time of submission, but not required. If VELJ subsequently requests any archival of data sets and necessary supporting materials, the author will be expected to permit this.

Those desiring additional information click here; those with any concerns regarding restraints imposed by an institutional review board please email your concerns to: velj.submissions@gmail.com

CELS 2018--Paper Submission Deadline Extension

The University of Michigan Law School and the Society for Empirical Legal Studies (SELS), co-hosts of the 13th Annual Conference on Empirical Legal Studies (CELS) on November 9-10, 2018, in Ann Arbor, Michigan, are pleased to announce that the deadline for paper submissions has been extended until June 29, 2018 (midnight, Eastern Time).

CELS is a highly regarded interdisciplinary gathering that draws researchers from across the United States and the world, bringing together scholars in law, economics, political science, psychology, policy analysis, and other fields who are interested in the empirical analysis of law and legal institutions. Papers are selected through a peer review process and discussion at the conference includes assigned commentators and audience questions.

CELS will consider empirical papers spanning all areas of empirical legal studies, including areas of newer or evolving interest, such as replication studies. Authors are encouraged to submit works-in-progress; however, submissions should be completed drafts that include principal results. Submitted papers must be unpublished (and expected to be unpublished at the time of the conference). If accepted, authors will have an opportunity to submit a revised draft prior to the conference for presentation and discussion. Please note that accepted papers will be made available to all conference participants.

The conference will include a poster session that features not only traditional-style posters presenting original research but also a "book nook" in which authors or editors of recently published (or forthcoming) volumes may exhibit their work and its primary findings. Book authors or editors who are interested in the "book nook" should contact the organizers at: CELS-2018@umich.edu

To submit a paper, please go to the CELS 2018 Conference page on SSRN --  https://hq.ssrn.com/conference=CELS-2018 (or here). (Do not use the regular SSRN submission site.)

For more information about CELS 2018 please visit the conference website (here).

Navigating Output Among Stata, Excel, and Word

Trying to accomplish seemingly simple tasks like moving, in tabular form, output among Stata, Excel, and Word platforms strikes many as something that should not be close to impossible. Regrettably, even such seemingly simple tasks can get complicated (and frustrating) quickly. Stata Corp is helpfully deploying summer intern talent to such challenges and concrete products from this effort are already emerging. By updating one command (tab2xl), creating another (tab2docx), and leveraging a third (putdocx), users can begin to more easily move tables across software platforms. This Stata Blog post summarizes the necessary moves and includes helpful illustrations.