Applying Social Psychology to Prevent Careless Responding during Online Surveys

Abstract

A major threat to data quality in online surveys is careless responding (CR; Meade & Craig,) or insufficient effort responding (e.g. Bowling, Huang, Bragg, Khazon, Liu, & Blackmore,). In three studies, we use social psychological theories to develop and test three prevention strategies (Ward & Pond III,) related to increasing respondent motivation to respond carefully. Study 1 presented control, scripted, or video-recorded instructions designed to increase the social influence of survey administrators on survey participants. Participants in the control group were significantly more likely to admit to CR than the script and video groups. Compared with the control, scripted instructions decreased interest, and had no effect on objective indicators of CR. Study 2 found that instructions designed to induce cognitive dissonance increased logical consistency of responses and survey interest. Instructions to create a sense of hypocrisy increased accuracy on instructed-response items. Study 3 showed that leveraging social exchange theory in survey instructions generally had no effect on CR. Similar results were found for both continuous and dichotomous scoring of indicators of CR across the three studies. Results demonstrate that facets of CR can be influenced via survey design. Future studies are needed to develop a more thorough understanding of best practices in survey design with respect to preventing CR.