Empirical Legal Studies

Northwestern University Law Review--2nd Annual Empirical Legal Scholarship Issue: Call for Papers

The Northwestern University Law Review is pleased to announce its second annual issue dedicated to empirical legal scholarship, to be published in Spring 2020. We welcome pieces making use of any and all empirical tools—including qualitative, quantitative, and mixed methods—to illuminate and engage questions of legal interest.

The Northwestern University Law Review accepts empirical articles on an exclusive basis only. The exclusive submission window for the 2020 empirical issue will run from March 15–April 15, 2019. Participating authors must agree to withhold the manuscript from submission to any other publications until receiving a decision from us. All publication decisions will be released no later than July 31, 2019.

Interested authors must submit articles and essays via email to Empirical Articles Editor Kerri Howard (at: kerri.howard@nlaw.northwestern.edu).

Please submit the article as a .doc or .docx file with a cover letter or similar email; CV; and, if desired, supporting materials. More information about submission requirements and the empirical selection process is available here.

CELS 2019 -- Call For Papers

Spearheaded by CELS 2019 co-presidents David Bjerk, Eric Helland, and Dan Krauss, Claremont McKenna College and the Society for Empirical Legal Studies (SELS) are pleased to announce the 14th Annual Conference on Empirical Legal Studies (CELS) on November 15-16, 2019, in Claremont, California.

CELS is a highly regarded interdisciplinary gathering that draws researchers from across the United States and the world, bringing together scholars in law, economics, political science, psychology, policy analysis, and other fields who are interested in the empirical analysis of law and legal institutions. Papers are selected through a peer review process, and discussion at the conference includes assigned commentators and audience questions.

CELS will consider empirical papers spanning all areas of empirical legal studies. Authors are encouraged to submit works-in-progress; however, submissions should be completed drafts that include principal results. Submitted papers must be unpublished (and expected to be unpublished at the time of the conference). If accepted, authors will have an opportunity to submit a revised draft prior to the conference for presentation and discussion. Please note that accepted papers will be made available to all conference participants.

For general information about the CELS 2019 conference (click here); to submit a paper (click here).  Please note the paper submission deadline: June 21, 2019 (midnight, Pacific Time).

"Significance" in Marginal Effects v. "Original" Model

A lively discussion (here) on the Stata list takes up the question: "This variable has significant effects in the original (logit/probit/heckprobit/whatever) model. But its marginal effect is not significant. Why?"

As the ensuing (and on-going) discussion makes clear, this nuanced discussion involves, among other things, careful attention to precisely what specific hypotheses are being tested in a margins specification versus an "original" specification.

Data on Post-Conviction Criminal Expungement

As briefly described in a recent NYT op-ed (on March 20, 2019; link here for those who can get behind the NYT pay-wall), Professors J.J. Prescott (Mich.) and Sonja Starr (Mich.) submit an increasingly discussed policy--a push for increased expungements of criminal convictions--to data. As the paper notes, motivating the increased policy attention to expungements are concerns about "reentry barriers faced by people with criminal records." Contributing to an under-developed empirical literature on the possible impacts of criminal expungements is a paucity of public information on expunged criminal records. In Expungement of Criminal Convictions: An Empirical Study, Prescott and Starr lever their access to "complete, deidentified criminal records from the Michigan State Police on all individuals who had obtained criminal record “set-asides” (Michigan’s term for record-sealing) as of March 2014, as well as full criminal history records for much larger comparison groups of individuals with convictions that were not set aside."

The paper's careful attention to research design addresses one (but not all) traditional threat to identification posed by selection bias. By adopting a within-person design--and not comparisons between those received expungements and those who did not--the specification's identification is not distorted by the likely (almost assured) systematic differences between expungee and non-expungee sub-groups. That said, even the paper's within group design can not fully blunt potential distortions owing to variations in the timing of an individuals' expungement application. A summary of key findings is included in the abstract.

"Laws permitting the expungement of criminal convictions are a key component of modern criminal justice reform efforts and have been the subject of a recent upsurge of legislative activity. This debate has been almost entirely devoid of evidence about the laws’ effects, in part because the necessary data (such as sealed records themselves) have been unavailable. We were able to obtain access to deidentified data that overcomes that problem, and we use it to carry out a comprehensive statewide study of expungement recipients and comparable non-recipients. We offer three key sets of empirical findings. First, among those legally eligible for expungement, just 6.5% obtain it within five years of eligibility. Drawing on patterns in our data as well as interviews with expungement lawyers, we point to reasons for this serious “uptake gap.” Second, those who do obtain expungement have extremely low subsequent crime rates, comparing favorably to the general population—a finding that defuses a common public-safety objection to expungement laws. Third, those who obtain expungement experience a sharp upturn in their wage and employment trajectories; on average, within two years, wages go up by 25% versus the pre-expungement trajectory, an effect mostly driven by unemployed people finding jobs and very minimally employed people finding steadier or higher-paying work."

SCOTUS, Flowers v. Mississippi, Social Science & Cornell Law School News

My CLS colleague, Sheri Johnson, is scheduled to argue Flowers v. Mississippi in the Supreme Court tomorrow. In addition to falling into Sheri’s scholarly (and lawyerly) sweet-spot, post-conviction capital defense appeals, this particular case (and appeal) has generated significant public attention flowing in part from a recent In The Dark podcast episode (Season 2: Curtis Flowers; click here). Tomorrow’s argument will engage with issues related to the prosecutor's peremptory challenges and possible Batson problems.

And were that not enough, another CLS colleague, Valerie Hans, invested some time in her Social Science in the Law course this semester on an array of underlying complex social science and legal dimensions raised by the Flowers litigation. (For a flavor of the issues covered click here.)

2019 Northwestern-Duke Causal Inference Workshops

Northwestern and Duke law schools, along with the Society for Empirical Legal Studies, will co-sponsor the 10th annual Research Design for Causal Inference workshops. Co-organized by Bernie Black (Northwestern) and Matthew McCubbins (Duke) and hosted at Duke University School of Law, in Raleigh-Durham, NC, the Main workshop is set for August 12-16, 2019, with an Advanced workshop following on August 19-21, 2019 (also at Duke). As the workshop overview notes, "Research design for causal inference is at the heart of a 'credibility revolution' in empirical research. We will cover the design of true randomized experiments and contrast them to natural or quasi experiments and to pure observational studies, where part of the sample is treated in some way, the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will assess the causal inferences one can draw from a research design, threats to valid inference, and research designs that can mitigate those threats." To these ends, the Main workshop will cover the basics of causal inference, including potential outcomes, matching; difference-in-differences, instrumental variables, and regression discontinuity designs. The Advanced workshop will focus on principal stratification (a generalization of causal IV analysis), advanced matching with panel data, and application of machine learning to causal inference. Both workshops feature an outstanding faculty and provide an excellent and efficient way to become acquainted with contemporary approaches for making causal inferences from various kinds of observational and experimental data. For those seeking additional registration information click here.

Longitudinal Study of Lawyer/Life Satisfaction: UVa Law Class of 1990 Over Time

Building on an earlier survey (conducted in 2007) of Univ. of Virginia Law School students (class of 1990) 20 years out, in Lawyers at the Peak of Their Careers: A 30-Year Longitudinal Study of Job and Life Satisfaction, forthcoming in JELS (Vol.16:1), John Monahan (UVa) and Jeffrey Swanson (Duke) re-surveyed (conducted in 2017) those responding individuals 30 years out to gain some sense of how professional and life satisfaction may have evolved over time. Interacting the 2007 and 2017 response rates suggests that 58 percent of the (still living) UVa Law class of 1990 participated in both survey waves. While questions about how much one may reasonably generalize from a self-selecting sample of a single graduating class from a single elite law school certainly persist, that such longitudinal survey work is unusual evidences the degree of difficulty incident to such research endeavors. A summary of key findings follow from an excerpted abstract.

"We found respondents to have taken diverse career paths, with no single work setting accounting for more than one‐quarter of the respondents and with fully one‐third of the respondents changing jobs in the past decade. Marked gender differences in the professional lives of respondents persisted (e.g., women continued to be much more likely than men to forego full‐time employment 'in order to care for children' (30 percent vs. 4 percent)). Working conditions at large private law firms stayed problematic, with the portion of respondents negatively affected by a stronger stress on economic sustainability being twice as high among those working in large firms (77 percent) than among those working in other settings (38 percent). Finally, both career satisfaction and life satisfaction again were found to be high, with 77 percent of respondents satisfied with the decision to become a lawyer, and 91 percent satisfied with their lives more broadly."

2019 ICPSR Summer Programs/Workshops

Information on the 2019 ICPSR Summer programs and workshops is now available.

FOUR-WEEK SESSIONS:
Held in Ann Arbor, the Summer Program’s two separate Four-week Sessions (June 24 - July 19; July 22 - August 16) provide an immersive learning experience and select from more than 40 courses, including regression, Bayesian analysis, longitudinal analysis, game theory, MLE, SEM, causal inference, machine learning, multilevel models, race/ethnicity and quantitative methods, and more.

SHORT WORKSHOPS:
For those needing to learn a specific methodological technique in just a few days, the Summer Program offers more than 40 short workshops in 9 cities. Workshops of interest include:
  *   Advanced Multilevel Modeling with HLM (May 28-31, Amherst)
  *   Qualitative Comparative Analysis (June 17-19, Ann Arbor)
  *   Multilevel Modeling in the Social Sciences (June 17-21, Chapel Hill)
  *   Experimental Methods in the Social Sciences (June 17-21, Ann Arbor)
  *   Regression Discontinuity Designs (June 24-26, Ann Arbor)
  *   Advanced Topics in Dynamic Panel Models (July 2-4, Vancouver)
  *   Spatial Econometrics (July 8-12, Ann Arbor)
  *   Regression Analysis for Spatial Data (July 15-19, Boulder)
  *   Practical Data Science and Data Management (July 29-August 1, Ann Arbor)
  *   Machine Learning: Uncovering Hidden Structure in Data (July 29-August 2, Berkeley)
  *   Bayesian Multilevel Models (August 12-16, Berkeley)
  *   Causal Inference for Clustered Data (August 19-23, Berkeley)

For more information click here.

"heckman" v. "etregress"

Two-stage modeling if not uncommon in ELS and relevant Stata coding has become quite specialized for subtly distinct contexts. A recent discussion on the Statalist (click here) provides a helpful (if brief) clarity.

The question posed sought modeling advice on estimating "the effect of having a female director on board on earnings management. I have a continuous variable for earnings management as the dependent variable. I want to first predict the probability of having a female board member versus having an all-male board. I am doing this to mitigate the sample selection bias."

The discussion elicits a distinction between the proper use of the "heckman" and "etregress" two-stage commands. While the "heckman" command is used when you observe outcomes for only the treated variable of interest, " the "etregress" command, by contrast, is used when "some outcome is observed for all units."

Advances in Data Selection: Evaluating Police Conduct

While agreement on a common set of metrics facilitates longitudinal studies as well as comparisons across studies, these benefits should not deter fresh new thinking about new sources of data that inform lingering research questions. A new paper by Aurelie Ouss (Penn--criminology) and John Rappaport (Chicago), Is Police Behavior Getting Worse? The Importance of Data Selection in Evaluating the Police, illustrates this point nicely. In it, the authors' analysis of police conduct draws from traditional data sources, including litigation levels and municipal settlement payouts, as well as a relatively new data source: liability insurance claims.

Exploiting 23 years worth of liability insurance data, involving 350 law enforcement agencies and a single insurance carrier, the paper finds that "while lawsuits and [settlement] payouts have trended upwards over the past decade, insurance claims have declined." What the authors conclude is that: "... police behavior is not getting worse; rather, public responses to policing harms are intensifying."

Stata Graphic "Trick" of the Day

For too many (myself included), Stata graphics can be among the more complicated coding tasks. Thus, concrete coding tips for some of the more "exotic" graphics capabilities invariably warrant attention (even if only to satisfy curiosity). In this spirit I pass along a brief discussion concerning coding requirements for a "fan" chart in Stata (here). In an example of what is possible (below), the presentation of a standard "line" chart is graphically enhanced with +/- 1, 2, and 3 standard error "cones."

Conference on Data Science and Law

Prof. Elliott Ash (ETH-Zurich) passed along information on a conference submission deadline extension for the Conference on Data Science and Law, hosted by ETH-Zurich, Switzerland, on 7-8 June 2019. The conference’s submission deadline is now: 12 Feb. 2019. An excerpted conference description and related information follow.

“Interest is growing among scholars from a variety of disciplines and across the globe in research at the intersection of data science and law. In recent years, the digitization of legal texts and developments in statistics and computer science have paved the way for new methodological approaches in this area…. The inaugural Conference on Data Science and Law will convene researchers across disciplines who are interested in this burgeoning field. Scholars in law, social and behavioral sciences, digital humanities, computer science, machine learning, and data analytics are invited to present works-in-progress in all areas of empirical legal studies that involve big data sources or use data science techniques, including natural language processing, machine learning, algorithmic fairness, topic modeling, or network analysis."

For those interested, paper proposals can be submitted directly to: dasa.skopp@gess.ethz.ch, with an email subject line: “Conference Submission”.

Database of State Tort Law Reforms (v.6.1)

Ronen Avraham (Tel Aviv/Texas) recently alerted me to an update of one of the more helpful resources around for those who study torts and tort reform. Specifically, the (DSTLR v.6.1) "updates the DSTLR (5th) and contains the most detailed, complete and comprehensive legal dataset of the most prevalent tort reforms in the United States between 1980 and 2018. The DSTLR has been downloaded more than 2700 times and has become the standard tool in empirical research of tort reform. The dataset records state laws in all fifty states and the District of Columbia over the last several decades...." For direct access to the dataset (in Excel) click here.

Power and More Power (Testing)

Power and sample-size calculations are an important part of planning a scientific study. In addition, too often papers that report (perhaps unexpected) null results do not take the next step and assess the possibility that any null results may be ascribed to an underpowered sample. While simple power tests are understood by many (click here for an inventory; click here for a helpful general intro video), power testing for more complex empirical specifications are not. A recent entry (the first of a series) on the Stata Blog (here) walks readers through power testing for more sophisticated modeling.

Marginal Effects Primer

Prof. Richard Williams (Notre Dame) put together an exceedingly helpful "primer" on Stata's "margins" command. Prof. WIlliam's PowerPoint slide deck (here) walks readers through the basics, provides helpful explanations, and illustrates though an array of concrete examples. As marginal effects gain in popularity many might find this helpful.

UPDATE: A loyal Els Blog reader, Prof. Jeff Yates (Binghamton), alerted me to two addition helpful links germane to margins (here and here).