Study Finds Probability-Based Surveys To Be Most Accurate
The research industry has raised doubts about the accuracy of non-probability based online surveys since their inception, even while their use has grown. A new peer-reviewed study by Stanford University researchers, Professor Jon Krosnick, David Yeager, and their co-authors, provides an extensive comparison of seven non-probability Internet survey panels with two other survey platforms based instead on probability sampling. One is based on KnowledgePanel® from Knowledge Networks, and the other probability-based solution uses random-digit-dialing telephone methodology. The authors concluded that the non-probability Internet surveys were less accurate, and customary weighting adjustments did not uniformly improve them.
Knowledge Networks provides you access to the study and the supplement via the links below.
Prof. Krosnick presented these findings at a KN breakfast forum in the Boston area; below is the full video and selected clips.
Presentation Clips - Windows Streaming Video
- Moving from Telephone to Internet (00:41)
- Indicator of validity #1: LinChiat Chang's experiment: Mode comparison with sample held constant/latent variable analysis (02:29)
- Indicator of validity #2: Systematic measurement error: Response order effects (01:53)
- RDD recruitment with Internet survey administration: Description of new analysis (03:19)
- Weigh constructs and why they matter/unweighted comparisons among panel types (03:17)
- Internet survey accuracy/response rates (01:51)
- Assessing validity (Telephone vs. Internet results) (00:43)
- Estimating reliability via latent variable structural equation modeling (01:39)
- Internet data collection: Probability versus opt-in sample (results) (07:15)
- Has anything improved since 2004? (01:02)
Krosnick/Yeager/Javitz respond to questions about their study – ABCNews.com
Gary Langer, ABC News polling expert, has given over his most recent blog for Krosnick, Yeager and Javitz to answer questions about their much-discussed new paper. They give clear answers that shed additional light on the shortcomings and proper uses of opt-in polling. Click here to read what they had to say.