Online Opt-In Surveys, Sample Weights, and Accuracy

For some obvious and less-than-obvious reasons, polling in general and online opt-in polling in particular have come under increased scrutiny. Despite their increased popularity (owing partly to cost factors), a recent Pew Research Center Report notes that such surveys have "raised some concern within the industry because, while low participation rates pose a challenge for all surveys, the online opt-in variety face additional hurdles." In many instances researchers turn to various sample weighting adjustments in an effort to off-set sample biases. While the brief Report can speak for itself, one clear take-away is that: "Even the most effective adjustment procedures were unable to remove most of the bias." Interestingly, "when it comes to accuracy, choosing the right variables for weighting is more important than choosing the right statistical [adjustment or 'sample weighting'] method." Andrew Gelman's (Columbia--Statistics) comments on the report are found here.