ICAR Research Data Repository for Knowledge Management
Replication Data for: How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It
Harvard Dataverse (Africa Rice Center, Bioversity International, CCAFS, CIAT, IFPRI, IRRI and WorldFish)
View Archive Info| Field | Value | |
| Title |
Replication Data for: How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It
|
|
| Identifier |
https://doi.org/10.7910/DVN/EZSJ1S
|
|
| Creator |
Montgomery, Jacob M.
Nyhan, Brendan Torres, Michelle |
|
| Publisher |
Harvard Dataverse
|
|
| Description |
In principle, experiments offer a straightforward method for social scientists to accurately estimate causal effects. However, scholars often unwittingly distort treatment effect estimates by conditioning on variables that could be affected by their experimental manipulation. Typical examples include controlling for post-treatment variables in statistical models, eliminating observations based on post-treatment criteria, or subsetting the data based on post-treatment variables. Though these modeling choices are intended to address common problems encountered when conducting experiments, they can bias estimates of causal effects. Moreover, problems associated with conditioning on post-treatment variables remain largely unrecognized in the field, which we show frequently publishes experimental studies using these practices in our discipline's most prestigious journals. We demonstrate the severity of experimental post-treatment bias analytically and document the magnitude of the potential distortions it induces using visualizations and reanalyses of real-world data. We conclude by providing applied researchers with recommendations for best practice.
|
|
| Subject |
Social Sciences
Post-treatment bias Causal inference Experimental design |
|
| Contributor |
Torres, Michelle
|
|