From the Perspective of Editors and Reviewers

While this particular paper and related post have little to do with ELS per se, Gelman's observations along with the post's comments may help authors understand how editors and reviewers discharge their functions. The paper's small sample (N=44) almost dooms it from the start. But, as the comments make clear, other problems emerge as well. Bottom line, if a paper's sample, methodology, research design, or analysis, etc., don't pass a "smell test" (basic persuasiveness which pivots to some degree on one's sense of replicability), most editors/journals will pass on it. (That the paper achieved publication--in The Quarterly Journal of Experimental Psychology--was itself insufficient to insulate it from important criticism.)