A Recap of the Empirical Legal Scholarship Workshop

As I stated earlier, I just attended the 3-day Empirical Legal Scholarship Workshop held at Northwestern in Chicago. It was great to see such diversity among the attendees in terms of substantive areas of interest, level of experience doing empirical work and understanding of statistics, and variety of law schools. The workshop is clearly geared for beginners, though it provides a good statistics refresher course for those of us who already engage in empirical work. (Though it looks like there will be an “Advanced” workshop held at Wash U next academic year that presumably will start to dive into the various complexities of multiple regression.  Andrew asked for suggestions on topics to cover.  My list included panel effects, rare events, graphics, time series, diagnostics, and predicted probabilities, among other items.)

 A number of basic and helpful tips were delivered at the workshop. For example: (1) Do not be an advocate for your hypotheses.  Always look for alternative accounts. (2) Write down every single way that you could measure or operationalize a concept. (3) Andrew Martin justified the use of STATA as the preferred stats software. Why STATA? More control over graphics; fewer limitations of stats models than SPSS; really cool modules for STATA; flexible between menu and commands; less $ than SPSS. But he made clear that ease is not one of the reasons for picking STATA.  (The ELS Blog reached a similar conclusion.) (4) If you take a stats course in the future be sure to learn the following: Diagnostics for outliers and leverage; Diagnostics for non-linearity; Diagnostics for incorrect distributional assumptions; Multi-collinearity; Models for different types of dependent variables.

Day one started with a discussion of the Sunstein et. al piece about ideological voting patterns on the courts of appeals, serving as the introduction for Lee Epstein’s conceptual overview of empirical work, research design, data collection and coding. Days two and three, mostly taught by Andrew Martin, was, as he called it, an overview of the first 2/3 of an intro stats class (inference, descriptive stats, cross-tabulations, linear regression, etc.), only introducing multiple regression. There were some classic objections to the Sunstein piece: the piece did not look at judicial reasoning in opinions, no operationalization of law, and frustration with the explanations of findings, though not as much discussion about inter-coder reliability as I would have expected.   In addition, a helpful paper “Coding Variables” by Epstein and Martin (Encyclopedia of Social Measurement, Vol. 1, 2005) was also distributed, in addition to several other helpful handouts on STATA commands, statistics vocabulary, and helpful references.  Overall, a worthwhile program.

Comments

[Jason Czarnezki]

Jeremy,

There was only very little discussion on experimental design (simply because it is rarely done in ELS) and ANOVA. But yes, the workshop discussed many statistics including one- and two-sample t-tests, chi-squares, r-squared, standard errors, standard deviations, and linear and multiple regression.

-jjc

[Jeremy A. Blumenthal]

Given Profs. Epstein and Martin's expertise in polisci research and regression techniques, I was curious whether the Workshop focused on other statistics and methodologies as well - in particular, experimental design and analysis, of which there is not much in current ELS. For instance, was there discussion of how to design an experiment, run t-tests, chi-squares, ANOVA's, contrast analysis, etc.? (To be crystal clear, I'm not questioning or criticizing anything about the Workshop or such research; I just wasn't there so I don't know!)

More broadly, do others have the sense that there is little such research in ELS today? If so, should there be more?

[William Henderson]

Jason, this is a teriffic post. I am filing it right now. bh.

[James]

I may not be the most sophisticated econometrician in the world (only an MA in econ), but working at the FTC has given me some great insights as to what ELS people should be more familiar with. When you are talking about more advanced topics I think it would be beneficial to learn more about not only specific techniques, but how to fix common statistical problems. One problem we come across frequently is endogenaety. As for advanced techniques, I think learing to use an "order statistics" approach would be helpful as well as a very detailed description of "difference in differences" analysis. That is my two cents, take it for what its worth. Oh, and it excites me to see that STATA is the preferred program. In my mind, it is far superior to anything else for analysis (if not data cleaning/manipulation, for which SAS is better suited)