Robert MacCoun (Stanford) and Saul Perlmutter (Berkeley) recently published a fascinating comment in Nature, Blind analysis: Hide results to seek the truth, in which the authors urge empiricists to employ “blinding” at all levels of study, including data analysis. As the authors note:
“Many motivations distort what inferences we draw from data. These include the desire to support one's theory, to refute one's competitors, to be first to report a phenomenon, or simply to avoid publishing 'odd' results. Such biases can be conscious or unconscious. They can occur irrespective of whether choices are motivated by the search for truth, by the good mentor's desire to help their student write a strong PhD thesis, or just by naked self-interest.
We argue that blind analysis should be used more broadly in empirical research. Working blind while selecting data and developing and debugging analyses offers an important way to keep scientists from fooling themselves.”