The Accuracy of Online Surveys with Non-probability Samples of People who Volunteer to do Surveys for Money

Online surveys of the general public are currently being conducted with two sorts of samples: (1) probability samples, recruited through RDD phone calls or face-to-face visits to respondents' homes, or (2) groups of people who are not selected from the population of interest by probability sampling and instead volunteer to do occasional on-line surveys for money (in response to online ads or email invitations). To compare the quality of data obtained from these two sorts of samples, the same questionnaire was administered simultaneously by an RDD telephone interviewing organization, a probability-sample online survey organization, and seven online survey groups that employ non-probability samples. Results from the different firms were compared in terms of the accuracy of distributions, the effects of manipulations, regression coefficients, and trends over time.