« Guest Blogger: John Pfaff | Main | The Polls - Modern Morals »

03 April 2006


Dan Kahan

This is a question that I and a team of other researchers have been investigating for a while. Our answer is: cultural cognition. As a result of various overlappling psychological and social mechanisms, individuals conform their view of the "facts" on disputed policies to their cultural evaluations of the behavior or laws in questions. We have data, from a national study of 1800 persons, that reltes this phenomenon to differences of views on gun control, death penalty, environmental regulation, etc. The data are summarized and papers uploaded at http://research.yale.edu/culturalcognition. (One paper that supplies a good overview is Cultural Cognition & Public Policy (http://papers.ssrn.com/sol3/papers.cfm?abstract_id=746508).
BTW, we used lay persons in our studies, but Hank Jenkins-Smith, a political scientist at the Bush Public Policy School at Texas A&M, has gotten similar results looking at scientists evaluating enviornmental policies.
Anyway-- I'll be guest blogging in June to present more information on this line of research.

Jeremy A. Blumenthal

In an article a couple years ago I've suggested that in many instances presenting meta-analyses is one way to address - not solve - the "dueling data" issue. When a quantitative synthesis of a particular body of research is presented and analyzed, it helps give a bigger picture view of that research and the accompanying issues.

More important, I think, it reduces the opportunity to cherry-pick among studies that support your viewpoint, encouraging academics and policy-makers to "speak the same language" - the language of what the overall research synthesis shows.

Finally, moderator analysis can help identify what it might be about different studies in a particular field that led them to their different conclusions.

Sara Benesh

I really should be working on my Midwest paper right now, but...

I completely agree with Chris. When I was "guest blogging" a while back, I noted that I am extremely uncomfortable with normatives -- this is exactly why. Once our inquiry is aimed at "proving" or "disproving" a notion or showing that some kind of behavior is "good" or "bad," we lose the science. In science, we need to formulate theories and hypotheses and test them and then see what happens and what normative implications might follow. We have to be, in other words, *agnostic* as to what we might find. If we can do that, and be honest about reporting it (b/c, as we all know, statistics lie -- or, at least, can be forced to lie) we'll be closer to being able to really "know" something about our subject. I do think the peer review process helps separate the wheat from the chaff, even if one does sometimes draw "bad" reviewers.

Okay, back to work!

Chris Zorn

Laura hit on something that, to my mind, is the most frustrating things about ELS: The tension between (American) legal and scientific cultures. Simply: Legal academics typically take an adversarial approach to scholarly inquiry that is at best inconsistent with (and at worst in direct opposition to) the more positivist/Popperian/whatever "scientific" culture in most of the social sciences. I saw it regularly at NSF, where law faculty would regularly submit grant proposals that (to paraphrase) started, "If you give me this grant, I will prove that...".

An adversarial perspective on inquiry and scholarship is fine. Better than fine: It is necessary and valuable, both for the pedagogical function law schools play and as one means of getting at normative and other kinds of questions that are not especially amenable to scientific thinking. At the same time, I'd suggest that the biggest challenge the ELS movement (are we a movement? I don't know...) faces is to reconcile these two complementary-yet-in-tension ways of thinking about legal scholarship.

Laura Beth Nielsen

These are excellent points. I think the review process (or lack thereof) in law reviews is partly to blame. When student-editors want to do a symposium on some topic, they think they have to have "balance" (not a bad idea when you are unable to separate good from bad scholarship).

Seeking balance can result in the clusters of articles you describe -- some on each side with little or no analysis of the quality of the different studies. This makes it easy for non-empiricists to say, "see, we just don't (or can't) know" and then move the argument to the normative. Eventually in law, we have to get to the normative, but it can be done on the basis of empiricism.

Don't we have the standards in each discipline? Maybe the trick is teaching them to legal academics so they can better evaluate research? And, convincing them the standards are meaningful for figuring out some empirical fact about the world.

We are used to taking any evidence that helps our argument, twisting it to our purpose and moving on. We have to be willing to say, "hey the (good) science proves me wrong." Or, in the alternative, "this science supports my point but it is done so badly that it is not credible evidence of my point."

So, in addition to actually learning the standards (which is hard, but do-able), there might need to be some culture shift in how we argue in law -- it is very different than how we argue in our disciplines (and much more empirically-based). Such a culture shift, obviously is much harder than simply teaching legal academics about standard deviations and margins of error.