Record Details

Replication Data for: How Much Should We Trust Instrumental Variable Estimates in Political Science? Practical Advice Based on 67 Replicated Studies

Harvard Dataverse (Africa Rice Center, Bioversity International, CCAFS, CIAT, IFPRI, IRRI and WorldFish)

View Archive Info
 
 
Field Value
 
Title Replication Data for: How Much Should We Trust Instrumental Variable Estimates in Political Science? Practical Advice Based on 67 Replicated Studies
 
Identifier https://doi.org/10.7910/DVN/MM5THZ
 
Creator Lal, Apoorva
Lockhart, Mac
Xu, Yiqing
Zu, Ziwen
 
Publisher Harvard Dataverse
 
Description Instrumental variable (IV) strategies are widely used in political science to establish causal relationships, but the identifying assumptions required by an IV design are demanding, and assessing their validity remains challenging. In this paper, we replicate 67 articles published in three top political science journals from 2010-2022 and identify several concerning patterns. First, researchers often overestimate the strength of their instruments due to non-i.i.d. error structures such as clustering. Second, IV estimates are often highly uncertain, and the commonly used $t$-test for two-stage-least-squares (2SLS) estimates frequently underestimates the uncertainties. Third, in most replicated studies, 2SLS estimates are significantly larger than ordinary-least-squares estimates, and their ratio is inversely related to the strength of the instrument in observational studies---a pattern not observed in experimental ones---suggesting potential violations of unconfoundedness or the exclusion restriction in the former. We provide a checklist and software to help researchers avoid these pitfalls and improve their practice.
 
Subject Social Sciences
instrumental variables
meta analysis
 
Date 2024-02-17
 
Contributor Xu, Yiqing