Careless response and attrition as sources of bias in online survey assessments of personality traits and performance
MetadataShow full item record
The online survey is critical to organizations to efficiently collect large amounts of data about variables like performance. However, this data collection method comes with challenges. Careless responding (CR) and attrition in online surveys introduce measurement error, and can lead to several psychometric issues. Despite the common co-occurrence of CR and attrition, previous studies have looked at CR and attrition separately, which may miss their additive impact in online survey measurement. Additionally, research has focused on CR and attrition without attending to their relationships with other variables. The aims of our studies are to build the nomological network around CR and attrition, and to determine the effects of the co-occurrence of CR and attrition. We investigated (a) the extent to which personality traits relate to both CR and attrition, and (b) the extent to which CR and attrition affect estimates of the relationships between personality traits and performance. We found a relatively high base rate of carelessness (23%) and mid-study attrition (11%). Conscientiousness, extraversion, neuroticism, and agreeableness were related to survey attrition and carelessness resulting in significant differences between respondents who carefully completed the survey and those lost via attrition or careless response screening. A simulation study provided estimates of the extent of bias resulting from: 1) various amounts of attrition and carelessness, and 2) correlations between attrition and carelessness with personality traits. Although the magnitude of bias is modest in most cases, there was substantial bias (.15) in correlation estimates in some situations when screening for CR and attrition. Based on findings from the current studies and extant literature, we suggest ways researchers and practitioners can address CR and attrition to improve the accuracy of the measured relationships between variables like personality and performance, and enhance the defensibility of conclusions.
Showing items related by title, author, creator and subject.
Litten, Verity; Roberts, Lynne; Ladyshewsky, Rick; Castell, Emily; Kane, Robert (2019)Objective: The attraction, selection, and attrition model posits that individuals actively self‐select into vocational and educational environments based on their personality traits and values. The present study investigated ...
Ward, M.K. (2019)Purpose: Online surveys enjoy widespread use because of their speed, access and convenience. A disadvantage is the threat of careless responding (CR) defined as responses from people who have not given adequate attention ...
Using virtual presence and survey instructions to minimize careless responding on Internet-based surveysWard, Mary; Pond, S. (2015)Internet-based survey data inform knowledge creation in research and justify work activities in organizations. While there are advantages to online surveys, this mode of administration comes with its own set of challenges. ...