TY - JOUR
AU1 - Kotzian, Peter
AU2 - Stoeber, Thomas
AU3 - Hoos, Florian
AU4 - Weissenberger, Barbara E.
AB - Manipulation checks are a recommended for experimental accounting research. Usage of information gained by manipulation checks varies. In some studies, participants who failed the manipulation check are removed from the sample. Other studies report the results of the manipulation checks but still use the full sample. Some authors recommend removing participants who failed the manipulation check as a means to increase the power of the statistical analysis. Others warn that removing these participants endangers the randomization as a crucial precondition for gaining valid insights from experimental research. Until now, there is little research on how sensitive results react to exclusion of participants. The purpose of this paper is to study the effects of removing participants failing the manipulation checks on the evaluation of a hypothesis and the development of alternative usages of the information gained from manipulation checks.Design/methodology/approachBased on an analytical model and a simulation, the authors show how removing participants who fail the manipulation check affects experimental findings.FindingsSimulations show that statistical results and conclusions drawn from an experiment differ substantially, depending on whether participants who failed the manipulation check are removed from the sample. As the participants who are removed are no random sub-sample, but share a certain property, the experimental results react strongly, typically showing significant results, where there are actually none.Originality/valueThis paper is, to the best of the authors’ knowledge, the first to address the sensitivity of experimental results to removing participants who fail the manipulation check from the sample and the implications for the validity of conclusions drawn from experimental accounting research. This paper’s contribution is a better way of using information gained in the manipulation check in the statistical analysis of the experimental data.
TI - To be or not to be in the sample? On using manipulation checks in experimental accounting research
JF - Accounting Research Journal
DO - 10.1108/arj-06-2019-0128
DA - 2020-06-15
UR - https://www.deepdyve.com/lp/emerald-publishing/to-be-or-not-to-be-in-the-sample-on-using-manipulation-checks-in-X0OE1H2Gvc
SP - 469
EP - 482
VL - 33
IS - 3
DP - DeepDyve
ER -