Leveraging work design to prevent careless responding during online surveys

Abstract

Purpose: Online surveys enjoy widespread use because of their speed, access and convenience. A disadvantage is the threat of careless responding (CR) defined as responses from people who have not given adequate attention to an item. CR can create psychometric problems and preventing CR has been ineffective. Online surveys can be conceptualized as work, especially on platforms such as Mechanical Turk (MTurk), and work design literature exhibits robust relationships between enhanced work characteristics and performance. The current study tests a new prevention strategy based on principles of good work design. Design/Methodology: We conducted an online survey study with Mechanical Turk workers using an experimental design with random assignment to one of two conditions: control vs. enhanced work design (EWD; in which autonomy, feedback, and task significance were boosted). Outcome variables were multiple indicators of CR and attrition. Results: Initial results show those in the EWD condition took more time on the survey and answered more consistently across similar items, and were more willing to take additional measures. Limitations: By enhancing several aspects of work design simultaneously limits our ability to isolate effects of individual facets of work design. However, multiple work design variables co-exist in reality. Research/Practical Implications: The results suggest that work design principles could help reduce CR in online surveys and improve the quality of data. Originality/Value: This study was the first to apply work design principles to address CR. The results showed promise and also improved our understanding as to why CR occurs.