st: Problems with plots from generalized sensitivity analysis (GSA)

[Bertel Teilfeldt Hansen <statalistbertel@gmail.com>]

Dear Statalist,

I'm doing a paper about post-conflict elections wherein I use Masataka
Harada's excellent ado-program gsa to examine possible bias from
omitted variables. The program produces contour curves of generated
variables similar to those of Imbens' sensitivity analysis (2003) and
is described in detail in Harada (2012).

The problem is that my contour curves turn out fine (and, as hoped,
lie far away from other included variables), but the points
representing gsa's saved variables are often extremely scattered.

Some of them are very far from the observed variables, and some of
them are very close. Also, gsa seems to save points which have the
same correlation with the treatment, but very different correlations
with the outcome, implying that they should not have an equally
confounding influence on the treatment effect. This also happens the
other way around (i.e. same correlation with the outcome, but
different correlations with the treatment) and persists when the
precision-option is set very low (meaning that the accepted span of
influence for variables to be saved is narrowed).

A sample of the commands I run are included below (I run the newest
update of stata 12.1 on Windows XP and 7):

Halving the coefficient on the treatment:

- gsa outcome_binary treatment_continuous controlvar1 controlvar2
controlvar3 ... , vce(cluster unit_var) tau(0.08) nplots(10)
observation(1000) precision(1) scatter -

Making the treatment effect insignificant on the 0.1-level:

- gsa outcome_binary treatment_continuous controlvar1 controlvar2
controlvar3 ... , vce(cluster unit_var ) tstat(1.645) nplots(10)
observation(1000) precision(1) scatter -

I'm thinking the problem might be caused by me having very few
observations (I have between 60 and 70, depending on the
specification), but other than that, I'm all out of ideas. I hope you
can help!


Thanks for your consideration!

Best wishes

- Bertel

Ph.d.-student Bertel Teilfeldt Hansen
University of Copenhagen
Department of Political Science
Email: bth@ifs.ku.dk
Phone: +45 35324104


Litterature

Imbens, Guido W. 2003. “Sensitivity to Exogeneity Assumptions in
Program Evaluation.”
The American Economic Review 93(2):126–132.

Harada, Masataka. 2012 "Generalized Sensitivity Analysis." Working
paper available here:
http://home.uchicago.edu/~masa/docs/Harada-GeneralizedSensitivityAnalysis.pdf
.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/