In the spotlight: Double-robust treatment effects
(two wrongs don't make a right, but one does)
If you ever wanted an extra shot at getting your treatment-effects model right, teffects can help you.
teffects allows you to write a model for the treatment and a model for the outcome. We will show how—even if you misspecify one of the models—you can still get correct estimates using doubly robust estimators.
In experimental data, the treatment is randomized so that a difference between the average treated outcomes and the average nontreated outcomes estimates the average treatment effect (ATE). Suppose you want to estimate the ATE of a mother’s smoking on her baby’s birthweight. The ethical impossibility of asking a random selection of pregnant women to smoke mandates that these data be observational. Which women choose to smoke while pregnant almost certainly depends on observable covariates, such as the mother’s age.
We use a conditional model to make the treatment as good as random. More formally, we assume that conditioning on observable covariates makes the outcome conditionally independent of the treatment. Conditional independence allows us to use differences in model-adjusted averages to estimate the ATE.
The regression-adjustment (RA) estimator uses a model for the outcome. The RA estimator uses a difference in the average predictions for the treated and the average predictions for the nontreated to estimate the ATE. Below we use teffects ra to estimate the ATE when conditioning on the mother’s marital status, her education level, whether she had a prenatal visit in the first trimester, and whether it was her first baby.
. webuse cattaneo2 (Excerpt from Cattaneo (2010) Journal of Econometrics 155: 138-154) . teffects ra (bweight mmarried prenatal1 fbaby medu) (mbsmoke) Iteration 0: EE criterion = 4.582e-24 Iteration 1: EE criterion = 5.097e-26 Treatment-effects estimation Number of obs = 4642 Estimator : regression adjustment Outcome model : linear Treatment model: none
| Robust | ||
| bweight | Coef. Std. Err. z P>|z| [95% Conf. Interval] | |
| ATE | ||
| mbsmoke | ||
| (smoker | ||
| vs | ||
| nonsmoker) | -230.9541 24.34012 -9.49 0.000 -278.6599 -183.2484 | |
| POmean | ||
| mbsmoke | ||
| nonsmoker | 3402.548 9.546721 356.41 0.000 3383.836 3421.259 | |
Mothers’ smoking lowers the average birthweight by 231 grams.
The inverse-probability-weighted (IPW) estimator uses a model for the treatment instead of a model for the outcome; it uses the predicted treatment probabilities to weight the observed outcomes. The difference between the weighted treated outcomes and the weighted nontreated outcomes estimates the ATE. Conditioning on the same variables as above, we now use teffects ipw to estimate the ATE:
. teffects ipw (bweight) (mbsmoke mmarried prenatal1 fbaby medu) Iteration 0: EE criterion = 1.701e-23 Iteration 1: EE criterion = 4.947e-27 Treatment-effects estimation Number of obs = 4642 Estimator : inverse-probability weights Outcome model : weighted mean Treatment model: logit
| Robust | ||
| bweight | Coef. Std. Err. z P>|z| [95% Conf. Interval] | |
| ATE | ||
| mbsmoke | ||
| (smoker | ||
| vs | ||
| nonsmoker) | -231.1516 24.03183 -9.62 0.000 -278.2531 -184.0501 | |
| POmean | ||
| mbsmoke | ||
| nonsmoker | 3402.219 9.589812 354.77 0.000 3383.423 3421.015 | |
Mothers’ smoking again lowers the average birthweight by 231 grams.
We could use both models instead of one. The shocking fact is that only one of the two models must be correct to estimate the ATE, whether we use the augmented-IPW (AIPW) combination proposed by Robins and Rotnitzky (1995) or the IPW-regression-adjust ment (IPWRA) combination proposed by Wooldridge (2010).
The AIPW estimator augments the IPW estimator with a correction term. The term removes the bias if the treatment model is wrong and the outcome model is correct, and the term goes to 0 if the treatment model is correct and the outcome model is wrong.
The IPWRA estimator uses IPW probability weights when performing RA. The weights do not affect the accuracy of the RA estimator if the treatment model is wrong and the outcome model is correct. The weights correct the RA estimator if the treatment model is correct and the outcome model is wrong.
We now use teffects aipw to estimate the ATE:
. teffects aipw (bweight mmarried prenatal1 fbaby medu) /// > (mbsmoke mmarried prenatal1 fbaby medu) Iteration 0: EE criterion = 2.153e-23 Iteration 1: EE criterion = 1.802e-26 Treatment-effects estimation Number of obs = 4642 Estimator : augmented IPW Outcome model : linear by ML Treatment model: logit
| Robust | ||
| bweight | Coef. Std. Err. z P>|z| [95% Conf. Interval] | |
| ATE | ||
| mbsmoke | ||
| (smoker | ||
| vs | ||
| nonsmoker) | -229.7809 24.96839 -9.20 0.000 -278.718 -180.8437 | |
| POmean | ||
| mbsmoke | ||
| nonsmoker | 3403.122 9.564165 355.82 0.000 3384.376 3421.867 | |
Mothers’ smoking lowers the average birthweight by 230 grams.
Finally, we use teffects ipwra to estimate the ATE:
. teffects ipwra (bweight mmarried prenatal1 fbaby medu) /// > (mbsmoke mmarried prenatal1 fbaby medu) Iteration 0: EE criterion = 3.901e-22 Iteration 1: EE criterion = 1.373e-25 Treatment-effects estimation Number of obs = 4642 Estimator : IPW regression adjustment Outcome model : linear Treatment model: logit
| Robust | ||
| bweight | Coef. Std. Err. z P>|z| [95% Conf. Interval] | |
| ATE | ||
| mbsmoke | ||
| (smoker | ||
| vs | ||
| nonsmoker) | -227.4408 25.62591 -8.88 0.000 -277.6667 -177.215 | |
| POmean | ||
| mbsmoke | ||
| nonsmoker | 3403.027 9.56025 355.96 0.000 3384.289 3421.765 | |
Mothers’ smoking lowers the average birthweight by 227 grams.
All of these results tell a similar story, so we assume that both the outcome and the treatment models are correct. When both models are correct, the AIPW estimator is more efficient than either the RA or the IPW estimator. We started off in search of robustness and ended up with extra efficiency.
References
- Robins, J. M., and A. Rotnitzky. 1995.
- Semiparametric efficiency in multivariate regression models with missing data. Journal of the American Statistical Association 90: 122–129.
- Wooldridge, J. M. 2010.
- Econometric Analysis of Cross Section and Panel Data. 2nd ed. Cambridge, MA: MIT Press.
—David Drukker
Director of Econometrics
