7 Causal inference: IV (instrumental variables) and its limitations
Some casual discussion
Ben Balmford:
… IV really is pretty unreliable
So (ConsistencyInferenceInstrumental?) Young, 2019 by Young highlights that the point estimates seem to not differ that much from OLS estimates… which i think suggests that IV is not (generally) dealing with the problem of OVB that it claims to.
DR: My concerns tend to be more:
With heterogeneity IV is not capturing the average effect, and often finds results that are very much driven by a particular subsample, often accentuating the supposed bias
Many IV’s are not defensible and testing the identifying assumptions is basically impossible
BB:
And then (brodeurMethodsMatterPhacking2018?) (link) is I think quite a neat comparison of IV and other methods. The general point being that IV seems to have more publication bias than the other approaches to causal inference. Not sure that is that much of a surprise. In my chat with one of the authors he pointed out that in IV any changes are less obvious - in DiD (which doesnt perform great either)/RD the assumptions are much clearer to visualise on a graph. I also think that - in kind of a related way - there are many more places where there is [researcher] discretion in the IV approach.
BB: Whereas in DiD (which doesn’t perform great either)/RD the its really easy to just graph the basic data and you can kind of eyeball that to see if there is any effect and if the assumptions are met (I actually think this kind of speaks somewhat for the need to really be able to test the assumptions at the heart of DiD and RD - it’s so hard to provide a valid statistical test that just the fact we can eye-ball test the assumptions is useful). DR: Don’t always trust your eyeballs!
7.1 Instrument validity
- Exogeneity vs. exclusion
- Very hard to ‘powerfully test’
IV not credible? Note that for an instrument to be valid it needs to both be exogenously determined (i.e., not selected in a way related to the outcome of interest) and to also not have a direct effect on the outcome (only an indirect effect through the endogenous variable
7.2 Heterogeneity and LATE
Basic consideration: what does IV identify and when:?
Focusing on a binary endogenous ‘treatment’ variable
- With heterogeneity
- With imperfect compliance
- With one-way compliance
7.3 Weak instruments, other issues
- With a ‘weak instrument’ … why does that matter?
7.4 Instrumenting Interactions
In randomized controlled trials it sometimes arises the issue that one is particularly interested in a potentially endogeneous explanatory variable and its interaction with the experimental treatments, that are exogenous by construction. From the stata list and mostly harmless econometrics >If you have \(X_1\) endogenous and an exogenous instrument \(Z\) , and your model includes two terms involving \(X_1\) and \(X_1 \times X_2\), then you should use two instruments: \(Z\), and \(Z \times X_2\).