Re: st: Problems with plots from generalized sensitivity analysis (GSA)

[<masa@uchicago.edu>]

This program -gsa- has not been uploaded on SSC, but I answer 
here because the question is posted to statalist, I will 
submit the program in a few month and the future users can 
refer to this post in the near future. -gsa- is currently 
available at my personal website with a few known minor bugs.
--------------------------

Hi Bertel,

I looked at your data and command, and tried to produce a good 
contour by myself. In this case, you are right. The small 
number of observation is the sole reason why you do not get a 
good contour. 

I felt that you understand what each option does correctly and 
indeed set precision at the small value (1). This does reduce 
the variations of the plots due to the error |t-\tilde{t}|, 
but does NOT reduce those due to the uncertainty of the 
contour, from which your analysis suffer.

As the coefficient has both point estimate and standard 
errors, the contour has point estimate (in line) and 
variances. This is because of the fact that, with actual data, 
the size of omitted variable bias can be different from the 
value calculated from the canonical formula of OVB depending 
on the control variables. Although the correlation between 
pseudo unobservable and control variables is near zero, the 
correlation conditional on the treatment variable may not.

First thing you might want to do is to drop control variables 
that are only weekly correlated with the treatment and outcome 
variables. Indeed, -gsa- produced much better contour with 
your data with a few control variables. Another thing is to 
use multiple imputation and to use the most conservative 
contour to defend your claim if your data have many incomplete 
cases. 

If you still have the problem, how should you interpret the 
widely-dispersed scatter plots and the contour with wide 
margin? Because you usually do not know the sign and strength 
of the correlation between pseudo unobservable and control 
variables conditional on the treatment variable, you might 
want to interpret such contour like confidence interval. That 
is, first drop 5% of the contour that are closest ("in terms 
of what?" would be another question which I have not figured 
out), and use the line that connects the plots closest to the 
origin as a lower-bound contour. In this way, you can trust 
the contour with 95% CI.

-- 
MASATAKA HARADA
masa@uchicago.edu
http://home.uchicago.edu/~masa/top.html
1-312-952-6124
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/