"Twisted Skepticism" or Ignoring Data At Your Own Risk

An interesting post (here) discusses the practice of "discounting" (often understood to mean: ignoring) empirical results, all in an effort to be "skeptical."  An excerpt:

"A vast number of scientists have managed to convince themselves that skepticism means, or at least includes, the opposite of value data. They tell themselves that they are being “skeptical” — properly, of course — when they ignore data. They ignore it in all sorts of familiar ways. They claim “correlation does not equal causation” — and act as if the correlation is meaningless. They claim that “the plural of anecdote is not data” — apparently believing that observations not collected as part of a study are worthless. Those are the low-rent expressions of this attitude. The high-rent version is when a high-level commission delegated to decide some question ignores data that does not come from a placebo-controlled double-blind study, or something similar."

The author goes on to note (with no small amount of irony) that: "These methodological beliefs — that data above a certain threshold of rigor are valuable but data below that threshold are worthless — are based on no evidence."

Comments

[Alan E. Dunne]

Thanks to Michael Heise and his cited author for pointing out that we should recognise that those who bomb and burn churches are simply taking practical measures to reduce the crime rate and mourn those denied the benefits of the portacaval shunt merely because the randomly controlled studies showed no benefit.

But it might be considered a kind of evidence that better evidence does so often give different results from the casual kind.

In the law of evidence followed by the courts, it is recognised that some evidence is just not worth the time and effort of considering. Again, so many different things can be proved by casual evidence.

The post "empiricism gone astray" quotes from its underlying criticism that "Counterintuitive empirical results are almost always wrong if not based on an appropriate empirical methodology.

[Anita]

Part of the discounting of data seems, in my opinion, to relate to excessive mathematisation (which is a wonderful device for reification through mystification).

I came to empirical legal studies as a reaction to the formalistic modelling of much law and economics. A colleague of mine, who is much more anthropological than I am, attended the last ELS conference and wrote the following report: http://www.paclii.org/journals/fJSPL/vol11no2/pdf/forsyth2.pdf The excessive mathematisation decreased accessibility of papers, and the gap between quantitative an qualitative, and the general absence of qualitative data were things she found the conference missed. That said, I find anthropologists very dimissive of anything quantitative.

Maybe the point is that the true methodological challenge for ELS, if it wants to avoid the pitfalls of inaccessibility and irrelevance to the majority of the world that does not 'speak math' is to build space for methodology that allows for explanation of data in plain English and the bringing together of the qualitative or anecdotal with the quantitative. This will allow it to act as a true bridge between a number of disciplines and not just law and statistics.

I keep an eye on the methodology section of this blog in the hope that this is discussed further by people wiser than me!