An interesting discussion recently emerged (here and here) of a study comparing two agricultural experiments (involving different seeds of cowpeas to farmers in Tanzania) -- one blinded, one unblinded. As Andrew Gelman (Columbia) notes: "Bulte et al. find much different results in the two experiments and
attribute the difference to expectation effects (when people know
they’re receiving an experiment they behave differently); Ozler is
skeptical and attributes the different outcomes to various practical
differences in implementation of the two experiments."
Dan Ho's (Stanford) interesting paper on restaurant grading, Fudging the Nudge: Information Disclosure and Restaurant Grading, appears in the current YLJ (122: 574, 2012) and will likely appeal to "foodies" (as well as those looking for a brief respite from grinding through fall semester grading). The paper exploits data from San Diego and New York and uncovers various structural challenges incident to information disclosure to consumers. The abstract follows.
"One of the most promising regulatory currents consists of “targeted” disclosure: mandating simplified information disclosure at the time of decisionmaking to “nudge” parties along. Its poster child is restaurant sanitation grading. In principle, a simple posted letter grade (‘A,’ ‘B,’ or ‘C’) empowers consumers and properly incentivizes restaurateurs to reduce risks for foodborne illness. Yet empirical evidence of the efficacy of restaurant grading is sparse. This Article fills the void by studying over 700,000 health inspections of restaurants across ten jurisdictions, focusing on San Diego and New York. Despite grading’s great promise, we show that the regulatory design, implementation, and practice suffer from serious flaws: jurisdictions fudge more than nudge. In San Diego, grade inflation reigns. Nearly all restaurants receive ‘A’s. In New York, inspections exhibit little substantive consistency. A good score does not meaningfully predict cleanliness down the road. Unsurprisingly, New York’s implementation of letter grading in 2010 has not discernably reduced manifestations of foodborne illness. Perhaps worse, the system perversely shifts inspection resources away from higher health hazards to resolve grade disputes. These results have considerable implications, not only for food safety, but also for the institutional design of information disclosure."
A paper now circulating in SSRN (and forthcoming in Judicature) explores the impact of Twombly and Iqbal on dismissal rates and does so with an interesting methodological twist. A New Look: Dismissal Rates in Federal Civil Cases, by Scott Dodson (Hastings), contributes to the growing empirical literature by coding at the individual claim--rather than the case--level. The abstract follows.
"In the wake of Twombly and Iqbal, a number of studies have been conducted to
determine the decisions' effects on dismissal practice in federal civil cases.
However, those studies have tended to code whole cases rather than claims --
leading to the ambiguous coding category of “mixed” dismissals and to problems
in characterizing the nature of the dispute -- and have failed to distinguish
between legal sufficiency and factual sufficiency, potentially masking important
detail about the effects of the pleadings changes.
This paper begins to fill in that detail. I compiled an original dataset of
district court opinions and coded each claim -- rather than whole case --
subject to an adjudicated Rule 12(b)(6) motion. For each claim, I also
determined whether the court resolved the motion on grounds of legal or factual
sufficiency. This methodology opened an unprecedented level of granularity in
The data reveal statistically significant increases in the dismissal rate
overall and in a number of subsets of claims. I also find an increase in the
relative prevalence and efficacy of factual-insufficiency arguments for
dismissal. Perhaps surprisingly, I find a decrease in the relative prevalence
and efficacy of legal-insufficiency arguments for dismissal. These data and
insights on the rationales of dismissals are new to the literature and suggest
that Twombly and Iqbal are affecting both movant strategy and judicial
Cornell colleagues, Ted Eisenberg and Marty Wells, empirically analyze leading ranking metrics for refereed law journals in their recent paper, Ranking Law Journals and the Limits of Journal Citation Reports. Their analysis of ranking outcomes emphasizes a pre-occupation with ordinal ranking and database bias. The abstract follows:
schools, scholars, and journals emphasize ordinal rank. Journal rankings
published by Journal Citation Reports (JCR) are widely used to assess research
quality, which influences important decisions by academic departments,
universities, and countries. We study refereed law journal rankings by JCR,
Washington and Lee Law Library (W&L), and the Australian Research Council
(ARC). Both JCR’s and W&L’s multiple measures of journals can be represented
by a single latent factor. Yet JCR’s rankings are uncorrelated with W&L’s.
The differences appear to be attributable to underrepresentation of law journals
in JCR’s database. We illustrate the effects of database bias on rankings
through case studies of three elite journals, the Journal of Law &
Economics, Supreme Court Review, and the American Law & Economics Review.
Cluster analysis is a supplement to ordinal ranking and we report the results of
a cluster analysis of law journals. The ARC does organize journals into four
large groups and provides generally reasonable rankings of journals. But
anomalies exist that could be avoided by checking the ARC groups against
citation-based measures. Entities that rank should use their data to provide
meaningful clusters rather than providing only ordinal ranks."
Noting that "In the United States, men are fifteen times as likely to be incarcerated as women," scholars have wondered whether this difference "can be explained by differences in criminal behavior or
circumstances, or are courts or prosecutors treating genuinely
equivalent cases differently on the basis of gender?" Given the obvious constitutional implications, reasons for this difference generate important policy and legal interest.
In Estimating Gender Disparities in Federal Criminal Cases, Sonja Starr (Michigan) explores reasons for this difference using a dataset that "traces federal criminal cases from arrest through sentencing. I find that gender gaps widen at every stage of the justice process and that men and women ultimately receive dramatically different sentences." An excerpted abstract follows:
"[the paper] finds large gender gaps favoring women throughout the sentence length distribution (averaging over 60%), conditional on arrest offense, criminal history, and other pre-charge observables. Female arrestees are also significantly likelier to avoid charges and convictions entirely, and twice as likely to avoid incarceration if convicted. Prior studies have reported much smaller sentence gaps because they have ignored the role of charging, plea-bargaining, and sentencing fact-finding in producing sentences. Most studies control for endogenous severity measures that result from these earlier discretionary processes and use samples that have been winnowed by them. I avoid these problems by using a linked dataset tracing cases from arrest through sentencing. Using decomposition methods, I show that most sentence disparity arises from decisions at the earlier stages, and use the rich data to investigate causal theories for these gender gaps."
In the complicated (and turbulent) education policy world, policymakers continue to fret over "gendered" outcomes, particularly, of late, in the STEM (science, technology, engineering, and mathematics) context. While such a topic is admittedly layered, research design issues typically frustrate studies of American schools. In Do Single-Sex Schools
Enhance Students’ Stem (Science, Technology, Engineering, and Mathematics)
Outcomes?, Hyunjoon Park (Penn-Sociology), Jere Behrman (Penn-Econ.), and Jaesung Choi (Penn-Econ.) exploit a unique data set from Korea where students are randomly assigned to either co-ed or single-gender high schools. Also interesting are the study's asymmetric findings. The paper's abstract follows.
"Despite women’s significant improvement in educational attainment, underrepresentation of women in Science, Technology, Engineering, and Mathematics (STEM) college majors persists in most countries. We address whether one particular institution – single-sex schools – may enhance female – or male – students’ STEM careers. Exploiting the unique setting in Korea where assignment to all-girls, all-boys or coeducational high schools is random, we move beyond associations to assess causal effects of single-sex schools. We use administrative data on national college entrance mathematics examination scores and a longitudinal survey of high school seniors that provide various STEM outcomes (mathematics and science interest and selfefficacy, expectations of a four-year college attendance and a STEM college major during the high school senior year, and actual attendance at a four-year college and choice of a STEM major two years after high school). We find significantly positive effects of all-boys schools consistently across different STEM outcomes, whereas the positive effect of all-girls schools is only found for mathematics scores."
While attention has focused on state medical malpractice over the years, attention has shifted away from claims against and payouts by physicians and surgeons. A recent paper by Myungho Paik (Northwestern), Bernie Black (Northwestern), and David Hyman (Illinois), The Receding Tide of
Medical Malpractice Litigation, exploits the National Practitioner
Data Bank and finds that per-physician payouts have fallen 46% below their 1992 level. The abstract follows.
"Tort reform has
been a hot issue during the past decade, as malpractice premiums spiked, and
state and federal legislators debated the desirability of damages caps. Nine
states adopted caps on non-economic or total damages during the period
2003-2006, joining twenty-two states that had previously adopted caps. Great
effort has been devoted to studying the impact of these caps, but overall trends
in claim rates and payouts have been ignored. Using the National Practitioner
Data Bank, we find the frequency of paid medical malpractice claims per
physician has been dropping steadily for almost 20 years, and is now less than
half the level it was in 1992. Payouts per physician have also been dropping
since 2003, and are now 46% below their 1992 level. The decline is largest in
states that recently capped total or non-economic damages, but there are also
large and sustained declines in states with older damage caps and states with no
damage caps. We identify several factors that may partially explain these
trends, and suggest possibilities for further research."
The Center for the Study of Law and Society invites applications for visiting scholars for 2013-2014. The Center fosters empirical research and theoretical analysis concerning legal institutions, legal processes, legal change, and the social consequences of law. Closely linked to the School of Law, CSLS creates a multidisciplinary milieu with a faculty of distinguished socio-legal scholars in sociology of law, political science, criminal justice studies, law and economics, legal history, law and psychology, and legal and social philosophy, along with visiting socio-legal scholars from the United States and around the world.
Among the benefits of being a visiting scholar at the Center for the Study of Law and Society are: library privileges at the Law School and at all campus libraries; invitation to our weekly Bag Lunch Speaker Series, workshops, and scholarly exchanges; UCB campus privileges, such as athletic facilities and permission to audit classes; and, when possible, assignment to shared or other office space and use of computer, internet, and other facilities. The Center will consider applications for periods of time that vary from one month duration to the full academic year.
The deadline for applications is November 16, 2012. More information, including full details about the application process, is available here: http://www.law.berkeley.edu/1273.htm.
In 2005, the National Research Council (NRC) evaluated the “More Guns, Less Crime” hypothesis using county-level crime data for the period 1977-2000. 17 of the 18 NRC panel members essentially concluded that the existing research was inconclusive on whether "right-to-carry" laws increased or decreased crime.
"We evaluate the NRC evidence, and improve and expand on the report’s county data analysis by analyzing an additional six years of county data as well as state panel data for the period 1977-2006. We also present evidence using both a more plausible version of the Lott and Mustard specification, as well as our own preferred specification (which, unlike the Lott and Mustard model used in the NRC report, does control for rates of incarceration and police). While we have considerable sympathy with the NRC’s majority view about the difficulty of drawing conclusions from simple panel data models, we disagree with the NRC report’s judgment that cluster adjustments to correct for serial correlation are not needed. Our randomization tests show that without such adjustments the Type 1 error soars to 44-75 percent. In addition, the conclusion of the dissenting panel member that RTC laws reduce murder has no statistical support.
Our paper highlights some important questions to consider when using panel data methods to resolve questions of law and policy effectiveness. Although we agree with the NRC’s cautious conclusion regarding the effects of RTC laws, we buttress this conclusion by showing how sensitive the estimated impact of RTC laws is to different data periods, the use of state versus county data, particular specifications, and the decision to control for state trends. Overall, the most consistent, albeit not uniform, finding to emerge from both the state and county panel data models conducted over the entire 1977-2006 period with and without state trends and using three different specifications is that aggravated assault rises when RTC laws are adopted."
While I truly hope not a sign of the times, as the title implies, Just Post It: The Lesson from Two Cases of Fabricated Data Detected by Statistics Alone, illustrates how simple statistical tests can be used to detect falsified data. Interestingly, Uri Simonsohn's (Penn-Wharton) main point involves a plea for journals to make available raw data used to support findings in published papers. That such a policy may assist in rooting out fraudulent papers (and deterring such unethical conduct) is a positive spillover. The abstract follows.
"I argue that journals should require authors to post the raw data supporting their published results. I illustrate some of the benefits of doing so by describing two cases of fraud I identified exclusively through statistical analysis of reported means and standard deviations. Analyses of the raw data provided important confirmation of the initial suspicions, ruling out benign explanations (e.g., reporting errors; unusual distributions), identifying additional signs of fabrication, and also ruling out one of the suspected fraudster’s explanations for his anomalous results. If we want to reduce fraud, we need to require authors to post their raw data."
"Despite the fundamental role of deterrence in the theoretical justification for medical malpractice law, surprisingly little evidence has been put forth to date bearing on its existence and scope. Using data from the 1979 to 2005 National Hospital Discharge Surveys and drawing on an extensive set of variations in various tort measures (e.g., damage caps) and malpractice standard-of-care rules (Frakes 2012a), I estimate a small and statistically insignificant relationship between malpractice forces and two metrics of health care quality emphasized by the Agency for Healthcare Research and Quality: (1) avoidable hospitalization rates (reflective of outpatient quality) and (2) inpatient mortality rates for selected medical conditions. At most, the evidence implies an arguably modest degree of malpractice-induced deterrence. For instance, at one end of the 95% confidence interval, the lack of a non-economic damages cap (indicative of higher malpractice pressure) is associated with only a 4% decrease in avoidable hospitalizations."
A recent paper now circulating on SSRN, The Innocent Defendant’s Dilemma: An Innovative Empirical Study of Plea Bargaining’s Innocence Problem, intrigues for an array of reasons. The subject matter--defendants' willingness to plead to offenses they did not commit incident to a plea bargain--certainly provokes. Also notable is the research design employed. The researchers, Lucian Dervan (S. Ill.) and Vanessa Edkins (psychology, Florida Tech.), constructed an experiment in which college students actually felt they were "accused of cheating and presented with her own incentives to admit wrongdoing and save the university the time and expense of proceeding before a disciplinary review board." The study involved "dozens of college students, and taking place over several months ... revealed that plea bargaining’s innocence problem is not isolated to an obscure and rare set of cases." Finally, insofar as this experiment involved human subjects, that the researchers managed to get their research project through IRB struck me as something of a minor miracle. An excerpted abstract follows.
"Strikingly, the authors’ study found that over half of the participants were willing to falsely admit guilt in return for a perceived benefit. This finding not only brings finality to the long-standing debate regarding the possible extent of plea bargaining’s innocence problem, but also ignites a fundamental constitutional question regarding an institution the Supreme Court reluctantly approved of in 1970 in return for an assurance it would not be used to induce innocent defendants to falsely admit guilt."
The Court's recent ACA decision propelled attention to perceptions of Court legitimacy. Erik Girvan (Oregon) pointed me to a post by a pair of political scientists at Duke, Chris Johnston and D. Sunshine Hillygus, who consider "Given that the Court ruled in favor of the constitutionality of the ACA bill, we were interested to see how Court legitimacy would vary across Democrats and Republicans. Would Democrats show higher levels of legitimacy because of the ruling? Conversely, would Republicans show higher legitimacy in spite of the ruling, because the Court remains relatively conservative overall?"
According to Johnston and Hillygus, "the present data support the conclusion that Supreme Court legitimacy is, to a meaningful extent, conditional on political predispositions as they relate to the output of the Court; however, what matters is how citizens perceive the Court’s output, and whether those perceptions are in line with their own preferences or not."
Yale Law School will begin offering a Ph.D in law, with the first incoming class arriving in Fall 2013. I learnd of this from a WSJ story; the Yale press release is here. From the latter:
"Because the level of the scholarship expected of entry-level law professors has risen quite dramatically, increasing numbers of law professors now pursue Ph.D.’s in allied disciplines like economics, history, philosophy, or political science. Because such disciplines train students in standards and questions that are different from those of the law, the natural next step for the legal academy is to create our own Ph.D. program that can focus on the questions and practices of the law itself. Students obtaining a Ph.D. in law may, of course, engage in interdisciplinary studies, but their work will be anchored in the framework of legal scholarship." (Dean Robert Post)
The program is clearly designed to prepare individuals for careers on law faculty; and, interestingly, the program is only open to individuals who have already received a J.D. This makes it an interesting move by Yale, both from a "possible pool of candidates" perspective and (more broadly) given some of the other changes in law schools recently (and their likely knock-on effects for faculty hiring). I'll have to mull this last bit over a bit more before commenting further, but my knee-jerk reaction is that this will further polarize the ranks of law school faculties with respect to credentals, focus, etc.
In related news, Gordon Silverstein, a political scientist formerly at (inter alia) Berkeley, will assume the position of Assistant Dean of Graduate Programs at YLS, effective Monday. He'll be in charge of the new Ph.D., as well as existing the LL.M. and J.S.D. programs. Congratulations, Gordon!