« Forum Post #1: Opening the Black Box of Law Review Selection | Main | Forum Post #3: Methodological Matters »

14 August 2007

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451b58069e200e3933bd2408834

Listed below are links to weblogs that reference Forum Post #2: Some Context from the Authors:

Comments

tadalafil

Under Article IV of the Constitution, which outlines the relationship between the states,

generic valtrex

hello friend thanks for sharing this post about Forum Post #2: Some Context from the Authors thanks !!!

Jason Nance

I also believe that there is a self-reporting bias, and, having a background in social science research, I can tell you that this problem permeates through several disciplines. Short of orchestrating some type of experimental design, it is something that social scientists have learned to live with. We do the best with the data that is available to us. We hope that survey participants are honest in their responses; and we hope that by keeping the survey anonymous, it will encourage honesty. I think it does for the most part. And, as long as that self-reporting bias is sufficient “random,” it shouldn’t affect the results too much.

I suppose we could address that fact in our study, but I think most readers are already generally familiar with that bias and take it into account as they evaluate our findings.

Dylan Steinberg

While I don't think Articles Editors are participating in a charade, I think the potential for self-reporting bias is real (and probably needs to be addressed head on in the paper, which it currently is not). There are two sources of a gap between reported results and "true" results that I think could have crept into our results.

The first I'll call an embarrassment bias: editors may be embarrassed to admit that they discard articles based on superficial criteria. Even though the survey was anonymous, I think there's some risk that editors will have underreported their reliance on certain criteria out of a sense that they "shouldn't" rely on them as much as they do.

The second potential source of bias is what I'll call an aspirational bias. If you ask an editor (as we did) how important the dgree of scholarly contribution of an article is, very few editors are going to say it's not important. But it's an open question whether student editors are really in any position to make a judgment on that question. I know that, when I was making article decisions, I paid a lot of attention to whether an article was making a scholarly contribution. But, truth be told, my perception of scholarly contribution and that of the actual scholars dealing with similar issues (i.e. the "true" perception of scholarly contribution) are almost certain to be different.

I don't think either of those factors makes our data less valid or interesting (in fact, if we're interested in how well articles editors do their job, we first have to understand what they think that job is and what factors they understand to be important), but I do think they both need to be considered when we look at what the results mean.

Bill Henderson

Dylan and Jason,

In the comments to the introduction, Michael Heise suggested that your study looks at what editors "say" influences the selection process rather the factors that actually make a difference. I know that Chris Zorn will further explore this important point.

But I am willing to go out on a limb and be skeptical that the alleged difference is large. I understand it is possible; but I think it is important to discuss the likelihood of such a disparity and our analytical basis for drawing this conclusion.

The selection process for Articles Editors ensure they possess a lot of naive intelligence. In addition, virtually every law review board has a meeting on the selection process that your described above; I have zero basis to believe that these folks are engaging in a charade. Articles editors and EIC's are generally smart, ambitious, and confident people who don't necessarily want to be someone else's patsy.

Absence a good story of self-delusion or strategic behavior (what is really at stake here?), I think it is worth considering the literal reading of the data.

Michael Heise

Dylan: Setting aside any technical issues with your (and your coauthor's) study, I want to commend both of you for even *thinking* about the issue seriously, let alone for undertaking the study. As law review editors it's not like you had a lot of free time to burn.

The comments to this entry are closed.

Conferences

November 2014

Sun Mon Tue Wed Thu Fri Sat
            1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30            

Site Meter


Creative Commons License


  • Creative Commons License
Blog powered by Typepad