Forum Post #6: Moving From Anecdote to Data

First, thanks to Bill and everyone for letting an empirical neophyte trespass on this blog for awhile.  As a typical junior professor, I am a regular submitter to law reviews and have spent untold hours talking to colleagues about the submission process.  However, lately these discussions have grown tiresome as I realize that as arbitrary and random as the process seems to be, unrelated anecdotes only make the process seem more random instead of creating some pattern of order.  No one person's "N" is ever going to be big enough for that person to say with any authority "The best way to get your article accepted at top journals is to do X."  However, these conversations seem always to dwindle to the point where Professor A says that doing one thing is important, but then Professor B will counter with the argument that the Professor A must be wrong because Professor B never does that one thing and always places well.  And so on.  So, I am pleased as punch to be moving from war stories to data.  If nothing else, the student-run law review system produced the two authors of this study and perhaps honed their writing skills to the point that they were able to advance knowledge in this way.  (I would like to hear about their own placement story, though, now that they are on the other side!)

However, as everyone else seems eager to point out, survey responses aren't the sort of data we need to conclusively dispel or confirm these anecdotes.  (As an alumnus of the Lee Epstein/Andrew Martin Conducting Empirical Legal Scholarship workshops, I remember one of them saying that the best way to introduce bias into your project is to do a survey!)  I won't belabor the point, but I think we all know that law review editors are smart enough to know what answers they should pick.  (However, a survey of former Articles Editors might produce less self-conscious responses.)  I think it's interesting that the "Tier 1" editors assigned lower importance scores to certain categories regarding author prestige and article topic, but those factors were still the most important in rank order.  And, as the authors note, when asked virtually the same question in different form regarding author's place in the world, editors answered differently depending on wording.  They don't care about how "notable" an author is, but they care whether the author is "highly influential in her respective field."

So, we know now that articles editors, even though they understand that they shouldn't say it's super important, admit that characteristics of the author are pretty much as important as characteristics of the work.  So, that seems to jive with everyone's worst fears and conventional wisdom.  And I think the objective data would support that.  A few weeks ago, I updated some armchair empiricism on which authors get published in the Harvard Law Review (not a respondent to the survey).  While my interest was in the gender breakdown of the authors published, I hasten to add that any casual legal scholar would recognize most of the names of those whose work was published in any given volume.  I would suspect the same would be true of at least the top 5 journals.

So, what can I possibly add to this forum?  Here are two small bits.  First, I think the "hot topic" factor here is not treated in a precise way.  Although the authors suggest that at least most journals do not look for hot topics, the factor was phrased as "The topic is one about which many articles are currently being written."  Well, that sentence does not cry out for agreement, and one can think of many other ways in which the same could be worded.  In fact, two factors that were ranked highly ("The article fills a gap in the literature" and "The topic would interest the general public") also describe hot topics.  One can imagine that a legal topic that is currently in the news would spawn articles that both fill a (new) gap and that would interest the general public.  Because of the timeliness, this topic may be the subject of many law review articles, but it's hard for an articles editor to know that at the beginning of the trend.

Second, one interesting aspect about the law review selection process is the agency problem.  Editors are choosing articles that they will work on, perhaps personally.  While the authors assume that these individual editors choose articles that will gain their law reviews citations and attention, in reality these editors will move on before this attention happens.  So, editors may be more interested in choosing topics that they themselves are interested in and possibly choosing authors that they would like to get to know.  I would be interested to know if any articles editors thought about the relationship between themselves and their authors.  I definitely remember the authors that I worked with.  And, I could definitely tell when I was getting an offer from an articles editor who really liked my topic.

Comments

[cheap viagra]

After a nearly fatal encounter with the Redeemer, Spawn's costume evolved into a more advanced form. The formerly immaculate shroud became ragged and shredded, and his costume lost its red coloring and became pure black and white.

[raivo pommer -eesti.]

In Deutschland können erstmals in der Nachkriegsgeschichte Aktionäre maroder Banken enteignet werden. Die Enteignung soll aber nur letztes Mittel sein, um beim angeschlagenen Immobilien- und Staatsfinanzierer Hypo Real Estate (HRE) eine Kontrollmehrheit des Staates zu erlangen.

Warum lässt der Staat die HRE nicht einfach pleitegehen?

Der Bund hat eine Insolvenz ausgeschlossen, weil die HRE mit einer Bilanzsumme von 400 Milliarden Euro zu wichtig für den Bankenmarkt ist. Sie ist ähnlich groß wie die amerikanischen Bank Lehman Brothers. Die Lehman-Pleite im Herbst beschleunigte die Finanzkrise, weil seitdem viele Wertpapiere unverkäuflich sind. Die HRE ist ein großer Herausgeber von Pfandbriefen. Sie sind neben Staatsanleihen die sichersten Wertpapiere der Welt. Dieser Ruf soll geschützt werden. Außerdem wickeln viele andere Banken und Länder Geschäfte über die HRE ab. Eine Insolvenz würde andere Institute mitreißen.

[raivo pommer -eesti.]

Mehdorn

Bundesverkehrsminister Wolfgang Tiefensee (SPD) hat sich gegen eine Übergangslösung an der Spitze der Deutschen Bahn AG ausgesprochen. „Ich glaube, dass wir jetzt eine Person brauchen, die die langfristige Strategie im Blick hat“, sagte er am Dienstag in der ARD. Die Gewerkschaften hätten zu recht darauf hingewiesen, dass es für die Personalie darauf ankomme, welche Ausrichtung der Konzern nach dem Rücktritt von Bahnchef Hartmut Mehdorn künftig haben solle.

Tiefensee betonte, dass es nach Ansicht der SPD „in der nächsten Legislaturperiode keine Fortsetzung der Teilprivatisierung“ geben sollte. Auch der SPD-Vorsitzende Franz Müntefering sagte am Dienstag: „Da ist auf absehbare Zeit überhaupt kein Gedanke mehr dran.“ Dies ergebe sich aus der aktuellen Situation, in der nicht ernsthaft an einen Gang an den Aktienmarkt zu denken sei, sagte er im Deutschlandfunk.

„Die Kanzlerin will keinen Schwarzen und die SPD keinen Roten“

Tiefensee versicherte, über die Nachfolge von Mehdorn werde nicht nach parteipolitischem Kalkül entschieden. „Die Kanzlerin will keinen Schwarzen und die SPD keinen Roten, sondern wir wollen gemeinsam einen Mann suchen oder eine Frau, die für diesen Posten geeignet ist.“ Die Gewerkschaften sollten in die Suche nach einem neuen Bahnchef eingebunden werden. „Wir brauchen jemanden, der stark ist, der hohe Fachkompetenz besitzt, der mit der Mitarbeiterschaft gut umgehen kann und der vor allem dieses Unternehmen mit der Politik zusammen durch eine schwierige Wirtschafts- und Finanzkrise führen kann und dann so aufstellt, dass wir in Europa punkten.“

[Bryan Camp]

Hi all, this is just one more anecdote to throw into the pot, but it supports the point that law review editors may well select articles based on their personal experience and interest. My first placement, at Washington and Lee during the August 1993 cycle, came when I was a practitioner at the IRS. The article was on RICO. If my memory serves me right, the main reason the article ended up at W&L was that the articles editor had worked on RICO that summer and got really excited about the paper because he knew and was interested in the topic I wrote on. I had a great experience with that journal and their editorial process.

It makes sense to me that an editor would be more likely to recognize the potential of an article to be a gap-filler or of wide interest when it is on a topic with which the editor has experience.

Regards, -bryan camp (now a prof. at texas tech law school)

[Katie Porter]

We attended in different years, but I thought that Epstein's point (I don't recall Martin making this statement) was that surveys **can** create bias so that if there is an objectively verifiable way to gather identical data that is preferable. The classic example is asking people what they weigh v. weighing them. To a certain degree, the research at issue wants to know what law review editors think--not necessarily what they do--and that requires a survey. If the purpose of the piece is to measure what editors do, the study is flawed. It should have analyzed the actual pieces and author criteria of articles that get published. But if the purpose was to measure what editors think, then a survey seems appropriate. I would disagree that alternate methodologies are inherently less biased. For example, empirical analyses of published opinions raise their own methodological issues; they are just less visible sometimes than survey bias. I certainly agree that the internal consistency of how editors responded to differently phrased querys on the same point raises concerns about the design of this particular survey, but I want to stick up for the potential of survey research to be useful.