The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions - PubMed
. 2016 Oct;111(4):493-504.
doi: 10.1037/pspa0000056. Epub 2016 Jun 13.
Affiliations
- PMID: 27295328
- DOI: 10.1037/pspa0000056
The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions
Haotian Zhou et al. J Pers Soc Psychol. 2016 Oct.
Abstract
The authors find that experimental studies using online samples (e.g., MTurk) often violate the assumption of random assignment, because participant attrition-quitting a study before completing it and getting paid-is not only prevalent, but also varies systemically across experimental conditions. Using standard social psychology paradigms (e.g., ego-depletion, construal level), they observed attrition rates ranging from 30% to 50% (Study 1). The authors show that failing to attend to attrition rates in online panels has grave consequences. By introducing experimental confounds, unattended attrition misled them to draw mind-boggling yet false conclusions: that recalling a few happy events is considerably more effortful than recalling many happy events, and that imagining applying eyeliner leads to weight loss (Study 2). In addition, attrition rate misled them to draw a logical yet false conclusion: that explaining one's view on gun rights decreases progun sentiment (Study 3). The authors offer a partial remedy (Study 4) and call for minimizing and reporting experimental attrition in studies conducted on the Web. (PsycINFO Database Record
(c) 2016 APA, all rights reserved).
Similar articles
-
Lumsden J, Skinner A, Coyle D, Lawrence N, Munafo M. Lumsden J, et al. J Med Internet Res. 2017 Nov 22;19(11):e395. doi: 10.2196/jmir.8473. J Med Internet Res. 2017. PMID: 29167090 Free PMC article.
-
When ab ≠ c - c': published errors in the reports of single-mediator models.
Petrocelli JV, Clarkson JJ, Whitmire MB, Moon PE. Petrocelli JV, et al. Behav Res Methods. 2013 Jun;45(2):595-601. doi: 10.3758/s13428-012-0262-5. Behav Res Methods. 2013. PMID: 23055167
-
Cooper ML. Cooper ML. J Pers Soc Psychol. 2016 Mar;110(3):431-4. doi: 10.1037/pspp0000033. J Pers Soc Psychol. 2016. PMID: 26963765 No abstract available.
-
Internet research in psychology.
Gosling SD, Mason W. Gosling SD, et al. Annu Rev Psychol. 2015 Jan 3;66:877-902. doi: 10.1146/annurev-psych-010814-015321. Epub 2014 Sep 22. Annu Rev Psychol. 2015. PMID: 25251483 Review.
-
Are we missing anything? Pursuing research on attrition.
Marcellus L. Marcellus L. Can J Nurs Res. 2004 Sep;36(3):82-98. Can J Nurs Res. 2004. PMID: 15551664 Review.
Cited by
-
Maradona in our minds: The FIFA World Cup as a way to address collective memory properties.
Bavassi L, Kaczer L, Fernández RS. Bavassi L, et al. Mem Cognit. 2020 Apr;48(3):469-480. doi: 10.3758/s13421-019-00983-9. Mem Cognit. 2020. PMID: 31823315
-
Leong V, Raheel K, Sim JY, Kacker K, Karlaftis VM, Vassiliu C, Kalaivanan K, Chen SHA, Robbins TW, Sahakian BJ, Kourtzi Z. Leong V, et al. J Med Internet Res. 2022 Jan 6;24(1):e28368. doi: 10.2196/28368. J Med Internet Res. 2022. PMID: 34989691 Free PMC article.
-
Lovell B, McCarty K, Penfold P, Wetherell MA. Lovell B, et al. Front Psychiatry. 2023 Sep 7;14:1192669. doi: 10.3389/fpsyt.2023.1192669. eCollection 2023. Front Psychiatry. 2023. PMID: 37743984 Free PMC article.
-
Lumsden J, Skinner A, Coyle D, Lawrence N, Munafo M. Lumsden J, et al. J Med Internet Res. 2017 Nov 22;19(11):e395. doi: 10.2196/jmir.8473. J Med Internet Res. 2017. PMID: 29167090 Free PMC article.
-
Shepherd L. Shepherd L. Sex Roles. 2019;80(1):25-40. doi: 10.1007/s11199-018-0912-x. Epub 2018 Apr 9. Sex Roles. 2019. PMID: 30651663 Free PMC article.
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Miscellaneous