Today's WSJ contributes reports on a recent paper (here) that seeks to contribute to a long-standing debate surrounding the proverbial "hot hand" fallacy. In their paper, Surprised by the Gambler's and Hot Hand Fallacies? A Truth in the Law of Small Numbers, Joshua Miller (Bocconi Univ.) and Adam Sanjurjo (Univ. de Alicante) suggest that a form of selection bias contributes to an underestimation of the true probability of an outcome when conditioned on a prior (and similar) outcome. Not sure if the paper works, but an intriguing potential wrinkle nonetheless.
While reasonable minds can (and do) differ on how best to legally operationalize the equal educational opportunity doctrine, if the doctrine is to stand for anything meaningful it has to at least mean that a child's educational "destiny" should be independent of the household income that a child happens to have been born into. Data, however, make painfully clear that on balance we fall short--far, far short--of this aspiration. Helping make these data "come alive" is an interactive graphic (in the New York Times) that allows users to "guess" at the relation between a child's chance at going to college and parental income levels. (After a user draws his/her own regression line onto the graph the user can then compare it with the actual regression line.) While depressing from a policy perspective it's a clever interactive graphic.
Empirical legal studies’ impact was on display last week in the Supreme Court’s death penalty decision, Glossip v. Gross. In particular, there is a lively back and forth among Breyer (joined by Ginsburg) in dissent, and the concurrences of Scalia and Thomas with no fewer than three JELS articles cited.
Suffice to say, Breyer and Ginsburg find the empirical evidence of problematic capital punishment application compelling, Thomas counters with questions about methods, and Scalia summarizes much of the scholarship as “abolitionist.” The link to the opinion is here. Cited article links follow.
The Supreme Court's recent Texas Department of Housing and Community Affairs v. The Inclusive Communities Project decision makes clear the broad point that the Fair Housing Act allows causes of action based on disparate impact claims rooted in statistical evidence. More specifically, however, the Court held that “[d]isparate-impact liability mandates only the removal of artificial, arbitrary, and unnecessary barriers” and that it is “not an instrument to force housing authorities to reorder their priorities.”
Consequently, what the decision means at a practical level remains anything but clear. Indeed, as William Consovoy observes at SCOTUSblog (here) "to defeat a disparate impact claim, housing authorities need only bring forth a 'valid interest served by their policies.' Furthermore, 'a disparate-impact claim that relies on a statistical disparity must fail if the plaintiff cannot point to a defendant’s policy or policies causing that disparity.' Indeed, the Court sent a strong signal that the disparate impact claim that gave rise to this dispute will have a difficult time moving forward under this standard."
This is a fascinating and complex topic where the rules/norms appear to be shifting in real-time. Some of the complexities are aptly illustrated in a recent post on Andrew Gelman's blog (here). (For those interested, even more valuable than the post itself is the growing list of comments that vividly illustrate this issue's potential complexities.)
Even the idea of fabricated data strikes serious academics as both absurd and profoundly offensive. A recent example involving survey research and a now-retracted Science publication (discussed here and here) serves as another regrettable reminder that, as I read somewhere, "In God we trust; all others bring data." If there is any silver lining to this sordid affair, it involves the scam's unraveling by graduate students seeking to replicate the original (bogus and discredited) findings.
"Using time diary and survey data from the Panel Study of Income Dynamics Child Development Supplement, the authors examined how the amount of time mothers spent with children ages 3–11 (N=1,605) and adolescents 12–18 (N=778) related to offspring behavioral, emotional, and academic outcomes and adolescent risky behavior. Both time mothers spent engaged with and accessible to offspring were assessed. In childhood and adolescence, the amount of maternal time did not matter for offspring behaviors, emotions, or academics, whereas social status factors were important. For adolescents, more engaged maternal time was related to fewer delinquent behaviors, and engaged time with parents together was related to better outcomes. Overall, the amount of mothers’ time mattered in nuanced ways, and, unexpectedly, only in adolescence."
Last week I have discussed the role of legal theory in ELS. Fishman's paper serves as a great relevant example regarding this same tension (Fishman, 2013).
In the previous post, I have suggested that the greater interconnection between legal theory and empirical legal studies could also help mitigate the criticism on the dramatic increase of ELS in certain legal communities. In today's post I will examine a question I should have probably mentioned earlier; why are so many Israeli legal scholars doing ELS and what can we learn from this fact regarding the likelihood of such increase in other countries as well.
I have not studied this question seriously, but it seems to me that relative to its size, there are more Israeli scholars in ELS conferences than any other non-U.S. country. (However, there is a long list of biases, availability being an obvious example, that would explain why my perception may be inaccurate.) This kind of observation has been shared in the past by Oren Gazal-Ayal from Haifa in the context of Law and economics (Gazal-Ayal 2007).
Like scholars from other countries, in order for Israeli scholars to be promoted, they need to be published in top U.S. journals (this is of course not the case in all countries and I actually think it’s the exact opposite when it comes to larger European countries’ legal academia). The need to be published in U.S. journals in order to get promoted is giving an obvious advantage to legal scholars who were educated in the U.S. Furthermore, the argument that Gazal-Ayal raises with regard to Law and Econ is that Israelis who want to participate in the global market of ideas, with special emphasis on the U.S. academic market, focus on law and economics, which tends to be a more universal area, relative to more doctrinal areas of research which are more jurisdiction dependent. One might wonder whether this might be the case with regard to empirical legal studies as well. In principle, many similarities can be identified; in both approaches the necessary knowledge of the doctrine is minimal (but see my previous post), the knowledge of math and/or statistics, respectively, could replace knowledge of U.S. law, and both partly rely on disciplinary fields which argue to be universal (e.g. economics or psychology). However, there are also some notable differences between these two communities of knowledge. First, in many strands of empirical legal studies, much of an argument's development process is related to collecting data about legally relevant institutions. In many of these institutions, variation among countries is such that when brought as evidence, it could never influence American legal policy without further “local” findings. Another strand of empirical legal studies is based upon experimental methods. Admittedly, in most aspects of experimental psychology, the country where the experiment was conducted in is not as important. In the experimental legal analysis, however, questions of cultural context might play a much larger role than in many other applied sciences.
Another two colleagues of mine, Ariel Bendor and Yifat Holtzman-Gazit, have written a paper in Hebrew in which they suggest that the focus on ELS is driven from the pressure universities in Israel put on researchers to get grants. In many of the universities in Israel getting grants is necessary for promotion. (In Israel there are 4-5 ranks in contrast to 2-3 in the U.S. so promotion is more present in the life of Israeli scholars.) Naturally, they argue, it is easier to get grants for empirical projects relatively to theoretical ones. While I personally would like to think that this is not the case, I have no empirical evidence against it.
In sum, one might conclude that if similar trends exist in other countries (e.g. being universal rather than writing on local law and getting grants), my colleagues' prediction might suggest that ELS is about to become more and more global.
As noted elsewhere (e.g., here), recent public fascination with (and debate about) the color of a dress hints at larger issues germane to law, including the efficacy and reliability of video evidence. A Slate magazine piece considers explanations for "visual ambiguity."
In recent years, more and more scholars define themselves as empirical legal scholars and it seems that this trend is only getting stronger in recent years (see, for example, Heise, 2002 ; George, 2006).
It seems that in the Israeli context, this phenomenon is even greater, for reasons I will discuss in one of my next blog posts. In my estimation, more than of 10% of legal scholars in Law faculties in universities in Israel define empirical legal studies as one of their main research topics. Some of them have even started their career as empirical legal studies scholars and some are “traditional” legal scholars who shift their focus to conducting empirical work. Being in the hiring committee of my own university and talking to friends in other universities, it seems that hiring empirical legal studies scholars is a priority.
On a personal level for me, that’s a dream come true. More people with interest in empirical legal studies, means more attention, more research budgets, more PhD students in the field and more scholars to collaborate with. Nonetheless, the main question I want to raise in this post relates to the optimal ratio of empirical legal scholars, in a given faculty or a given community of legal academics.
The obvious objection to such situation is the notion of diversity, where the need in multiple theoretical perspectives is important for a rich discussion among scholars and for exploring various views to any legal question. However, one might respond that in contrast to law and economics (which some raise objection to their dominance in Israeli academia), empirical legal scholarship doesn’t have to adopt a narrow theoretical standpoint, as for example, many of the scholars who attend the ELS conference come from diverse backgrounds. Moreover, taking a broader approach to ELS, that includes for example, scholars who use qualitative methods, would ensure an even larger theoretical perspective.
An additional criticism would be that in any law school there is a need for people who would study all legal doctrines. If too many scholars study law empirically, very few scholars will need to carefully read cases for their research. In a small country like Israel, this might translate into a reduction in ability of the legal academia to criticize and support the coherent development of the case law.
A third possible objection would be the legal expertise of such legal scholars. Accounting for the fact that people have limited time and energy to read and learn, almost by definition, a scholar who have to master all the methods of empirical legal research could not dedicate the same amount of effort and time to learn the legal doctrine.
A forth objection, might be the dominant community of knowledge. In many ELS sub-communities, such as in the area of decision-making and law, there are a few dozen legal scholars and several thousand psychologists and economists. Since one might assume that the interest of legal scholarship and other disciplines may not be always aligned, it is possible to speculate that the larger and more established community would be more dominant. While this is a desirable situation from a methodological perspective, it might carry some inadvertent effect for the ability of ELS to fulfill the needs of the legal community.
A possible solution to this question is related to a conversation I recently held with Hanoch Dagan, the former dean of TAU Law school, I will write about in the next post, on the disconnect between legal theory and empirical legal studies.
One key assumption typically shared by many assessing federal circuit court decisions is that the three-judge panels that hear cases have been randomly configured. Indeed, scores of scholarly articles have noted this 'fact,' and it has been relied on heavily by empirical researchers.
How circuit panels are configured is no longer merely an academic question. Adam Liptak, in today's New York Times, reports on a legal challenge pivoting on how the Ninth Circuit assembled its panels to hear important same-sex appeals.
While not necessarily squarely on point to all ELS Blog readers, a general discussion of social science ethics, re-ignited recently by an incident/study involving political scientists at Stanford and Dartmouth (for a description, click here) remains germane to many ELS scholars as well. In a recent post Andrew Gelman (Columbia--statistics) discusses suggestions by Macartan Humphreys (Columbia--Poli Sci) on how to think through the ethical dimensions incident to social science research in the field.
My Cornell colleague and leading constitutional law scholar, Mike Dorf, has an interesting and provocative post (here) that speaks to the array (and growing number) of state quarantine measures responding to the Ebola crisis.
The ELS angle, of course, is Mike's point (drawn from CDC data) that: "the log(viral load) just before symptoms develop is 4.6. A day later, the log(viral load) is 7.2. Thus, (assuming linearity to first order) 12 hours after symptoms develop, the log(viral load) is 5.8. That's a change of 1.2 in log(viral load), meaning that the change in viral load more than triples (because e to the 1.2 power is 3.32.)."
While not squarely in the typical ELS wheelhouse, the following excerpt just stopped me in my tracks.
"In her excellent book, Race to the Top, the journalist Elizabeth Green tells a story of a new hamburger that the A&W Restaurant chain introduced to the masses. Weighing 1/3 of a pound, it was meant to compete with McDonald’s quarter-pounder and was priced comparably. But the 'Third Pounder' failed miserably. Consultants were mystified until they realized many A&W customers believed that they were paying the same for less meat than they got at McDonald’s. Why? Because four is bigger than three, so wouldn’t ¼ be more than 1/3?"
To be sure, this degree of innumeracy is not typically present in law school classrooms (or I certainly hope not). That said, a general ambivalence (at best) or aversion (at worst) towards all things quantitative shapes the stream of students who self-select into law schools.