The single-firm event study design is a common one in the law and finance and securities litigation research literatures. One challenge to the standard t-statistic-based approach to inference involves a downward bias in asymptotic Type I errors. An interesting paper by Jonah Gelbach (Arizona), Eric Helland (Claremont McKenna College & RAND), and Jonathan Klick (Penn), Valid Inference in Single-Firm, Single-Event Studies, assesses this challenge and compares the standard approach to an alternative with the benefit of real-world data. An excerpted abstract summarizes the paper's findings.
"Our results show that the standard approach is plagued by systematic, downward bias in asymptotic Type I error rates relative to desired significance levels. We then offer a very simple but statistically sound alternative, called the SQ test. We show analytically that the SQ test’s asymptotic Type I error rate always equals the desired significance level. Using our CRSP data, we offer Monte Carlo evidence that in event studies with 99 pre-event observations, the SQ test performs very well at conventional significance levels. We then analyze the asymptotic power of the SQ test and the standard approach. The SQ test and the standard approach have the same size-corrected asymptotic power, which means that even when the standard approach is appropriate, there is no loss to using the SQ test. More relevant as an empirical matter, we show that the standard approach’s downward bias in asymptotic Type I error rates brings along severe power loss. As an empirical matter, then, use of the standard approach can be expected to lead to substantial anti-plaintiff bias in securities litigation, though either pro- or anti-plaintiff bias is possible as an analytical matter. By contrast, the SQ test has considerable asymptotic power, even against moderately sized fixed alternatives."