One wonderful thing about this blog is the opportunity to learn about various different methodological approaches available for data analysis. The workshops on methodologies and statistics (e.g. Northwestern's) are enormously helpful, and references for teaching and catching up on statistics have been as well. Experimental work is very important, and relatively uncommon approaches such as MDS, network analysis, propensity score analysis, and others can give good insight into what is going on with both conventional and unconventional data.
But I'd like to highlight one approach that cuts across many of these, and in my view is one of the most important contributions ELS can make. I'm a big fan of meta-analysis--of the quantitative synthesis of existing research studies--and advocate its use by ELS scholars, courts, agencies, practitioners, ice cream truck vendors, etc.
I've made some of my case for meta-analysis in a piece coming out in Temple Law Review, so I will be brief here. First, descriptively, meta-analysis is just that--analysis one level up. That is, primary research uses individual units (person, cases, courts, etc.) as the unit of analysis. The individual units that meta-analysis involves, though, are the empirical studies themselves from a particular body of research. Synthesizing the results of each study, the goals of meta-analysis are (1) to identify the presence or absence of an effect in an existing empirical literature; (2) to evaluate the strength of that effect, for instance by summarizing the average effect across a set or subset of studies; and (3) to identify moderator variables, elements of the various studies that might have reliably affected their outcomes. Thus, a good meta-analysis will identify and synthesize every study in a research area; find the average effect across all those studies in order to summarize the most current state of knowledge; and then systematically compare and contrast across studies, in order to identify methodological and substantive aspects of the various studies in a discipline that might be associated with the effect sizes they report.
Meta-analysis has a number of benefits, especially relative to the traditional narrative lit review: it is more comprehensive; it better avoids subjective judgments about what to include in a review; it is better at identifying "small" effects; it avoids over-emphasis on misleading statistical significance and p-values (focusing on effect sizes instead); emphasizes moderator variables; and, in my view, gives a better grounding for policy inferences.
Meta-analyses, unfortunately, are still rare in law reviews (Chris Guthrie and Dan Orr had one recently, but few others appear; they are cited sometimes, though). They are used in court, usually in mass tort cases, but are often misunderstood by courts and experts. My hope is that use of meta-analyses in ELS will increase--they serve well to summarize a body of research; they allow (or force) researchers and policy-makers to speak a "common language"--by making cherry-picking from existing studies more difficult--and, identifying moderator variables, they can prompt further research questions.