Empirical Legal Studies

Legal Impact Research & Evidence of Racial Profiling

Far too many legal and court scholars (as well as law students, lawyers, judges, etc.) simply assume, at least as a presumptive matter, that court decisions yield the outcomes they seek. Not enough recognize how institutional arrangements, incentives, the "law of unintended consequences," etc., inform how court decisions are implemented "on the ground." A recent paper, An Empirical Assessment of Pretextual Stops and Racial Profiling, by Stephen Rushin (Loyola-Chicago) and Griffin Edwards (UAB-Economics), provides yet another helpful and timely reminder of the efficacy of subjecting formal legal doctrines to data and rigorous research designs.

In their paper the authors exploit a change in Washington State law concerning police traffic stops. While a 1999 Washington State Supreme Court decision (Ladson) barred pretextual traffic stops in 2012 the Washington Court reversed course (Arreola) by permitting "mixed motive" traffic stops. Varying this legal rule opened a door for a DID research design that draws on 8.2M traffic stops in Washington State between 2008 and 2015. And what the results suggest is that the Arreola court decision "was associated with a statistically significant increase in traffic stops and searches of nonwhite drivers relative to white drivers. We find that most of the increase in traffic stop of non-white drivers after Arreola occurred during the daytime, when police officers could more easily ascertain a driver’s race." Overall, this paper will interest not only because of the massive dataset but also for its empirical specification.

Standard (OLS) Regression for a *Binary* Dependant Variable?

Evidently, the question is not as crazy as I likely would have assumed 40, 30, or, perhaps, even 10 years ago. In a widely-discussed draft paper (click here), Logistic or linear? Estimating causal effects of treatments on binary outcomes using regression analysis, Robin Gomila (Princeton--(grad student) social psych) makes a case for OLS. While Andrew Gelman (Columbia--statistics) is among those who disagree (here), he also recommends alternative parallel suggestions and approaches. To the extent that the Gomila paper is partly motivated by the sometimes "muddled" interpretation of coefficients after a logistic transformation, my quick sense is that altering the empirical strategy risks a potential over-reaction.

New Goodness of Fit ("GOF") Test

Testing whether two separate distributions systematically vary typically involves a t-test or ksmirnov test. These two popular GOF tests involve comparing two distributions' means (t-test) or a hypothesized equal distribution (ksmirnov). A recently-released user-written command in Stata, distcomp, moves the needle a bit for GOF testing by assessing the equality of the distribution functions point-by-point. That is, instead of a single rejection (or non-rejection), the distcomp test generates ranges of values in which the two distributions' difference may achieve statistical significance.

To locate and download distcomp use the "findit" or "ssc install" functions in Stata. For a much more thorough explanation and description of the distcomp command click here.

Variable "Skew"

Any discussion about what to do with a "highly-skewed" variable needs to immediately differentiate between "highly-skewed" dependent and independent variables. A "highly-skewed" (or "non-normally distributed") dependent variable (or, more accurately, as it relates to the errors) raises all the obvious questions when it comes to the appropriateness of an OLS regression specification given its underlying core assumptions. A highly-skewed independent variable, by contrast, raises none of these particular concerns. As one of the commentators notes (see discussion, and an illustration, here), in certain contexts a highly-skewed independent variable may simply reflect "just the way the data are."

"Inflated" Standard Errors

While what constitutes an "inflated" standard error remains contested, what is clear, however, is that there is no standard, "go-to," statistical "fix" for any "inflated" standard errors. As one commentator notes (see StataList discussion here), "they are what they are." Rather, a more helpful approach towards "inflated" standard errors is to assess what in the underlying data set (or model) might be driving the standard errors. The standard set of "usual suspects" includes, e.g., outliers in the data set, an "extreme" level of multicollinearity, or an "over-fit" model. (For more on "over-fitting" regression models, see here.)