« The Uses of Significance Testing | Main | Welcome to New Permanent ELS Blog Editor: Frank Cross »

30 January 2007

Comments

Ramon Henkel

Prof. P.K.Pattnaik:
The general rule is that the nature of the problem to be researched determines the range of methodologies which could be used. It is not unusual for methodologies used in one discipline to be used in other disciplines as well. I am not familiar with legal research and thus am ignorant of the types of problems researched and thus the appropriate methodologies. However, to determine if any social science research methodology is appropriate for a particular legal research problem, the researcher, such as yourself, need only consult an introductory level social science research methods text (there are many, many such) and see whether any of the methodologies presented are appropriate for the problem at hand.

Prof. P.K.Pattnaik

Dear Sir,
I like your issues. Is it safe to use social science research method in legal research? Please send some literature on empirical research method for legal studies
Prof. P.K.Pattnaik

Ramon Henkel

Professor Mohr's long comment makes the claim that in his book he believes he has formally demonstrated that it is impossible to develop social science theories, and this makes concerns about the criticisms of statistical inference from a philosophy of science perspective irrelevant. Since I have not read his book I cannot comment on the validity of his claim. As a substitute for my own reading, I did a Google search for critical reviews of the book, hopefully by philosophers of social science or others more versed in the issues of the possibility of social science theory than I. I found none so I have no sense of anyone's evaluation of the validity of his claim.
I do, however, agree with his position that social scientists (at least not sociologists) have not produced viable theory of the nature that exist in the physical sciences in spite of the attempts of many in the area of mathematical sociology and theory formalization during, roughly, the period from the 1960's through the the 1980's, not to forget the work by Stuart Dodd (Dimensions of Society) in the late 1930's--early 1940's.
In such a short commentary, Professor Mohr could not develop his perspective that the task of social science is to "establish what actually happened, and why, in the past in order that the knowledge gained might be helpful in the future" which seems to reduce social scientists to a subcategory of historians, thought perhaps more statistically oriented than most historians. I do not know if this is his intent as such brief statements are open to misinterpretation and misunderstanding. However, it seems to me that if all one is doing is describing a historical situation and teasing out what causal relations existed at that time, I see little value in applying statistical inference at all (this may also be Professor Mohr's position). A variety of descriptive techniques would certainly be useful in many of investigations, but, if I understand his position, there is no population to which a statistical inference might be made. Other interpretations of the probability model underlying significance testing might be made, but as noted previously, these interpretations have their own problems.

Lawrence Mohr

Longish comment: Henkel provides two types of reasons for why significance testing might be of little or no value in social science -- technical reasons and philosophical reasons connected to scientific inference. In the end, he concedes that the technical reasons are not fatal because they reflect misuses that can be overcome with proper training. I agree with this position and feel that I have observed a great many applications of testing that have not committed any of the technical errors he catalogs. He argues, however, that the scientific objections are serious and remain valid concerns relating to the utility of significance testing in the social disciplines.

I would urge that the philosophical objections are even less germane than the technical ones. The reason is that all of the philosophical objections depend on the premise that social scientists are engaged in developing theories and that testing is used, or misused, as a critical part of this process. In reality, although it is only too true that many social scientists are trying to develop valid theories of human behavior, they are not succeeding in doing that and never will succeed because it is impossible to develop such theories (unless they be strictly biological). Therefore, it is irrelevant whether or not statistical testing contributes to theory development. The question is whether it can help in the pursuit of other, more valid purposes of social research.

By "theories" I mean valid, stable, causal explanations of particular human behaviors, such as turning out to vote, adopting an innovation, being less conservative as a group than as a collection of individuals, and so forth. Typically, when we run regression models, for example, this is what we are trying to do. In my experience, leading social scientists will agree with the negative position taken above, no matter what they are doing with their research time. Moreover, if you propose to an individual, intelligent human being that you have a theory that will accurately predict whether he or she will adopt a particular innovation, even after knowing the theory, the person can laugh at you -- justifiably -- and proceed to defy you by violating the theory. Most important, however, I feel that I have proved this negative position formally in a recent book (The Causes of Human Behavior) under light assumptions to which most of us will subscribe. In the process, I also sought to demonstrate that the development of so-called probabilistic theories is just as futile an endeavor as deterministic ones.

As I see it, the purpose of social research is to establish what actually happened, and why, in the past in order that the knowledge gained might be helpful in the future. The emphasis in steering research plans, it seems to me, should be on this "helpful". It can be (but will not necessarily be) helpful, for example, to know what the electorate of a certain jurisdiction has been thinking and why. It might also be (but will not necessarily be) helpful to establish the causal explanation for the adoption or non-adoption of an innovation by a group of farmers in their real world or a group of freshman psychology students in the laboratory. It's up to us to design our projects with this kind of function in mind, and not in the attempt to develop theories, whether probabilistic or deterministic, such as are common in physics.

In such endeavors -- in trying to establish what happened in a particular past and why -- significance testing can definitely be of assistance, under the conditions that I offerred in my post, above.

Damon Cann

A recent article by Andrew Gelman and Hal Stern in the American Statistician points out a facinating irony: The difference between "significant" and "not significant" is itself not statistically significant.

http://www.stat.columbia.edu/%7Egelman/research/published/signif4.pdf

The comments to this entry are closed.

Conferences

July 2014

Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    

Site Meter


Creative Commons License


  • Creative Commons License
Blog powered by Typepad