Empirical Legal Studies

A Perspective on Measurement Error

Andrew Gelman (Columbia--Statistics) notes (here) that among statistics' three essential elements, "measurement, comparison, and variation," measurement receives short shrift. Why?

"Part of it is surely that measurement takes effort, and we have other demands on our time. But it’s more than that. I think a large part is that we don’t carefully think about evaluation as a measurement issue and we’re not clear on what we want students to learn and how we can measure this." To this I would add one additional practical aspect. For those conducting secondary analyses of data sets put together by others, most typically defer to measurement decisions already baked into data sets.

Regardless, when measurement goes awry, measurement error emerges and bad things happen.

CELS 2015 -- Call For Papers

As previously mentioned, CELS 2015 will be held at Washington University Law School, in St. Louis, on Oct. 30-13, 2015. Conference organizers, Adam Badawi, Rebecca Hollander-Blumoff, and Pauline Kim recently announced the Call for Papers (here). Please note the June 26, 2015, submission deadline.

Descriptive Statistics for Non-Excluded Cases

A small but interesting wrinkle. Many data sets include cases with missing data (hopefully not too many) and these cases will be excluded from many regression specifications. When "describing" the data set, does one use all of the cases (including those cases excluded from regression analyses) or just those cases included in the regression? If it's the latter, is there an easy way to generate basic summary statistics for the non-excluded cases? (The answer to the final question is "yes," and a helpful explanation--and illustrations--can be found here.)

Judicial Specialization: An International Perspective

Interest in specialized courts continues to grow, and some of this growth takes place outside of the U.S. To explore whether specialized courts achieve the goals claimed by proponents, a recent paper, Do Specialized Courts Make a Difference? Evidence from Brazilian State Supreme Courts, exploits variation in how Brazilian state supreme courts engage in constitutional review. In their paper, Carolina Arlota (Oklahoma) and Nuno Garoupa (Texas A&M) compare decisions from Brazil's non-specialized en banc courts and "specialized" court panels. An excerpted abstract follows.

"The dataset considered 630 cases of abstract review judged between January 1, 2006, and December 31, 2010, across twenty-five state supreme courts of the Brazilian federation. The main purpose of our inquiry is to determine whether or not there are significant variations in the outcome of the cases of abstract review as a function of a specialized panel. We find some evidence that the existence of specialized panels matters for the likelihood and rates of dissent as well as duration of procedures, but not for other variables. Implications for legal reform are also discussed."

Visual Displays of Quantitative Information

Compare the following two visual displays of quantitative information. The former is described by some (here) as a "bad chart." While perhaps reasonable minds can disagree, the later is generally recognized as "probably the best statistical graphic ever drawn" (here).

 

Practical Tips: Merging Files in Stata

Not infrequently researchers need to merge two separate data files into one. What should be an easy task is one fraught with tricky details. Structural issues (e.g., are you adding new "cases" or, rather, new data to existing cases) warrant initial attention as their resolution drives downstream ID file "linking" issues. Parts of this general issue are helpfully discussed in a thread (here).

Longitudinal Data on the Relation Between "Motherhood" Time and Child Outcomes

While admittedly only tenuously related to family law issues, broadly construed, the data set and results from a recent study, published in the Journal of Marriage and Family (77:355-72, Apr. 2015), caught my eye. In what is described as "the first large-scale longitudinal study of parent [here, mothers'] time" (more commentary here) on childhood outcomes, Does the Amount of Time Mothers Spend With Children or Adolescents Matter?, the authors (Melissa A. Milkie et al.) conclude that, overall and on balance, it matters far less than one might assume. An excerpted abstract follows.

"Using time diary and survey data from the Panel Study of Income Dynamics Child Development Supplement, the authors examined how the amount of time mothers spent with children ages 3–11 (N=1,605) and adolescents 12–18 (N=778) related to offspring behavioral, emotional, and academic outcomes and adolescent risky behavior. Both time mothers spent engaged with and accessible to offspring were assessed. In childhood and adolescence, the amount of maternal time did not matter for offspring behaviors, emotions, or academics, whereas social status factors were important. For adolescents, more engaged maternal time was related to fewer delinquent behaviors, and engaged time with parents together was related to better outcomes. Overall, the amount of mothers’ time mattered in nuanced ways, and, unexpectedly, only in adolescence."