« External Funding for Legal Scholarship | Main | Law School Rankings "Without the Clutter" »

07 April 2006

Comments

William Henderson

John,

This is a very good post. To my mind, your discussion essentially answered the question posed: Use the best available data, "properly disclaimed." (In my next paper, I am going to drop a footnote and give you attribution for that phrase.)

Here is a simple plan: (1) Identify the weaknesses in the data and check your assessments with an objective third party; (2) Explore possible methodological solutions (new methods often arise in response to data problems); and (3) if the results are amenable to alternative intepretations, some of which cannot be tested due to data limitations, clearly state that to be the case. Boom, we know more about the world.

Sure, a reader might seize on a statistically significant result and ignore unambiguous disclaimers on what that result might mean (it has happened to me). But why accommodate irresponsible or ignorant behavior? What is more troublesome is the prospect that some agency will not collect better data the next time around because a study getting us half-way there, and flagging the data issue, was never published.

In short, do honest work that assists the competent researcher, legislator, or lay reader.

PG

I wonder if there's also an inclination to aggregate data rather than divide it out and look at state-specific problems. For example, my understanding is that Congress passed the Violence Against Women Act because of a sense that some states weren't doing enough on this problem, but the data Congress put in the record mostly seemed to be about the total number of assaults and prosecutions, rather than targeting specific states for underenforcement. So we end up with Congress's dealing with traditional state issues, and the law journals helping them out by not looking at states on an individual basis. http://www.blogdenovo.org/archives/001274.html

The comments to this entry are closed.

Conferences

August 2014

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            

Site Meter


Creative Commons License


  • Creative Commons License
Blog powered by Typepad