"Checklists" have long-endured as an inexpensive--and seemingly efficacious--way to reduce risk flowing from tasks requiring complex decision-making. In many complex endeavors--e,g, piloting an aircraft or surgery--checklists are ubiquitous. Indeed, checklists are so pervasive that their efficaciousness is typically assumed or implied.
Exploiting a natural experiment that arose in Seattle and involved state department of health restaurant inspection protocols, in Do Checklists Make a Difference? A Natural Experiment from Food Safety Enforcement (JELS, 15:2) Daniel Ho (Stanford) et al. present results that challenge conventional wisdom. The abstract follows.
"Inspired by Atul Gawande’s bestselling Checklist Manifesto, many commentators have called for checklists to solve complex problems in law and public policy. We study a unique natural experiment to provide the first systematic evidence of checklists in law. In 2005, the Public Health Department of Seattle and King County revised its health code, subjecting half of inspection items to a checklist, with others remaining on a free-form recall basis. Through in-depth qualitative analysis, we identify the subset of code items that remained substantively identical across revisions, and then apply difference-in-differences to isolate the checklist effect in more than 95,000 inspections from 2001--2009. Contrary to scholarly and popular claims that checklists can improve the administration of law, the checklist has no detectable effect on inspector behavior. Making a violation more salient by elevating it from “noncritical” to “critical” status, however, has a pronounced effect. The benefits of checklists alone are considerably overstated."