Re: st: deriving a bootstrap estimate of a difference between two weighted regressions

[Steve Samuels <sjsamuels@gmail.com>]

To finish off this topic, perhaps:

I had occasion recently to examine the causal association between a
continuous outcome and a two category "treatment" variable.  The
"treated" group constituted about 2% of the sample. The method of
analysis was OLS, and propensity scores had been estimated by logistic
regression.  I computed bootstrap standard errors that included and
excluded the logistic step.

The result: standard errors for  average treatment effects ATE,  ATT,
and ATC  that included the logistic regression were roughly double(!)
the standard errors that did not.

Steve



On Wed, Aug 4, 2010 at 1:07 PM, Ariel Linden, DrPH
<ariel.linden@gmail.com> wrote:
> Thank you, Stas. I will take your suggestions under advisement!
>
>
>
> Date: Tue, 3 Aug 2010 14:37:51 -0500
> From: Stas Kolenikov <skolenik@gmail.com>
> Subject: Re: st: deriving a bootstrap estimate of a difference between two
> weighted regressions
>
> On Tue, Aug 3, 2010 at 1:20 PM, Ariel Linden, DrPH <ariel.linden@gmail.com>
> wrote:
>> Thank you Stas and Steve for your comments!
>>
>> When I stated that the first model's weight would be ATT and the next
>> ATC, it was already after running the propensity score model and
>> establishing
> the
>> weights for each subject:
>> ATT = cond(treatvar, 1, propvar/(1- propvar)), and ATC =
>> cond(treatvar, (1-propvar)/propvar, 1)
>>
>> Under these conditions, there should be no negative weights, so that
>> is
> not
>> a concern.
>
> The negative weights would come out of Steve's suggestion to entertain the
> difference in weights (as that's what your procedure boils down to).
>
>> I am thinking that the code would look something like this, but I
>> would appreciate your input:
>>
>> 1. bootstrap _b[treatvar] from first regression with [pw=ATT] 2. save
>> 10,000 samples to file (or tempfile) 3. bootstrap _b[treatvar] from
>> second regression with [pw=ATC] 4. save 10,000 samples to file (or
>> tempfile) 5. gen difference = treatvar1-treatvar2 6. bootstrap
>> r(mean): sum  difference, to get bootstrapped CIs
>>
>> Does this make sense?
>
> 1-2 will produce something very similar to the _se[treatvar] in your basic
> regression with ATT weights (probably with -robust- option), and
> 3-4 will produce something very similar to _se[treatvar] in the regression
> with ATC weights. I outlined the code for you in the previous message -- you
> need to bootstrap the whole estimation procedure = { the propensity
> regression (leading to the weights) + two main regressions with two sets of
> weights }. In other words, for each bootstrap sample, you would need to run
> everything in the curly brackets to produce your "difference" estimate.
>
> I cannot comment on the scientific validity of this procedure; other people
> more knowledgeable in treatment effect estimation could do that.
>
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>



-- 
Steven Samuels
sjsamuels@gmail.com
18 Cantine's Island
Saugerties NY 12477
USA
Voice: 845-246-0774
Fax:    206-202-4783

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/