A recent post by David Schwartz (Chicago-Kent)--wondering whether empirical legal scholars should shoulder "special ethical responsibilities"--ignited a fascinating (and timely) discussion over at Concurring Opinions. Two reasons prompt Schwartz's concerns. "First, nearly all law reviews lack formal peer review. The lack of peer review potentially permits dubious data to be reported without differentiation alongside quality data. Second, empirical legal scholarship has the potential to be extremely influential on policy debates because it provides 'data' to substantiate or refute claims. Unfortunately, many consumers of empirical legal scholarship — including other legal scholars, practitioners, judges, the media, and policy makers — are not sophisticated in empirical methods."
Schwartz's concern focuses on what he calls "weak data." By that he means "reporting [results from] data that encourages weak or flawed inferences, that is not statistically significant, or that is of extremely limited value and thus may be misused." Specifically, "[t]he precise question I have been considering is under what circumstances one should report weak data, even with an appropriate explanation of the methodology used and its potential limitations."
Whether you agree with Schwartz or not, he raises an important question that warrants attention.