« Do Attorneys Add Value?: Evidence From Tax Court Litigation | Main | APSA Law & Courts 2006 Article Prize »

15 May 2006

Comments

Tracy Lightcap

I think Gary King is right; we ought to either publish the exact probabilities for estimates or their confidence intervals and ditch the whole idea of conventional probablity standards. This is especially the case for the many, many studies that are not based on datasets drawn from a probability sample to begin with. For them reporting confidence intervals based on bootstrapped estimates would be much better.

And we'll probably get that right after the Revolution and not before. The difficulty seems to be that the journal editing boards are reluctant to accept papers using these methods (usually) and authors are reluctant to use them as a result. I find myself faced with this dilemma fairly often and I almost always end up opting for - yep - conventional standards and conventional probability estimates. OK, ok, I hear you: physician heal thyself! But that dog won't hunt when you want to publish in an increasingly competitive environment; why give the reviewers an excuse for ditching your work? I think what we need is a commitment by editors to actually preferring more exact standards.

Tonja Jacobi

The .05 standard is purely a product of convention, there is nothing objective about .05 being the standard, rather than say .01 or .10. Most people agree that you have to draw the line somewhere, and .05 captures the balance between avoiding type 1 and type 2 errors. But the suggestion here is that perhaps you do not need to draw the line: let the market determine for itself what articles get cited. I suspect that the .05 standard is now so entrenched that it would act as a powerful focus point anyway -- most academics would still value only those works that satisfy the .05 standard.
But the suggestion made by William of relying more on working papers, and forgetting about formal publication, has benefits beyond the external imposition of the artificial .05 standard, once again by letting the market determine the value of an academic contribution. This may be particularly apt for the law field, where almost everyone has a story of woe about their experiences with student editors. Certainly economics seems to be heading in that direction: papers are still published in final form, but most of the action happens when working papers are posted; but perhaps economists just have particularly high discount factors, and less patience for the lengthy publication period.

William Henderson

Tonja,

This is a very nice post. It relates to a post by Bill Ford on the de minimis market for studies that don't hit the conventional .05 p-value. See http://www.elsblog.org/the_empirical_legal_studi/2006/05/journal_of_spur.html

What inference do with draw from (a) the unwillingness of journals to publish studies that do not hit 5%, and (b) the reality that many published studies many not stand up very well with rigorous robustness test?

I almost think we would be better off just publishing everything as working papers. The brash ring would then be building knowledge rather then obtaining a good placement.

Stanford Law Review

The first article on the death penalty by Professors Donohue and Wolfers was published by the Stanford Law Review in response to Professors Cass Sunstein and Adrian Vermeule's article on the death penalty. The issue also includes an article by Carol Steiker and a response by Sunstein and Vermeule:

Cass R. Sunstein & Adrian Vermeule, Is Capital Punishment Morally Required? Acts, Omissions, and Life-Life Tradeoffs, 58 STAN. L. REV. 703 (2005), available at http://lawreview.stanford.edu/content/issue3/sunstein1.pdf.

Carol S. Steiker, No, Capital Punishment Is Not Morally Required: Deterrence, Deontology, and the Death Penalty, 58 STAN. L. REV. 751 (2005), available at http://lawreview.stanford.edu/content/issue3/steiker.pdf.

John J. Donohue & Justin Wolfers, Uses and Abuses of Empirical Evidence in the Death Penalty Debate, 58 STAN. L. REV. 791 (2005), available at http://lawreview.stanford.edu/content/issue3/donohue.pdf .

Cass R. Sunstein & Adrian Vermeule, Deterring Murder: A Reply, 58 STAN. L. REV. 847 (2005), available at http://lawreview.stanford.edu/content/issue3/sunstein2.pdf.

The comments to this entry are closed.

Conferences

August 2014

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            

Site Meter


Creative Commons License


  • Creative Commons License
Blog powered by Typepad