Re: st: Bootstrap error message

[Austin Nichols <austinnichols@gmail.com>]

Stas--
Alistair says he is aware of the result from Imbens that shows the
bootstrap is inconsistent for matching, so quite the opposite of what
you wanted him to cite.  Why anyone would pursue the bootstrap knowing
it will not produce correct SEs is beyond me.
He also claims that reweighting fails to achieve balance, which I find
implausible at best.

On Wed, May 11, 2011 at 11:21 AM, Stas Kolenikov <skolenik@gmail.com> wrote:
> Alistair Windsor is bootstrapping some complicated results of a
> propensity score matching procedure, and reports an obscure "e(b) not
> found" message.
>
> My reactions:
>
> If you have a -capture-, you should have -if _rc{ }- following it with
> treatment of the exception. Otherwise you just sweep the errors (that
> you obviously are expecting to occur) under the carpet. (In my code, I
> might leave an empty -else- and explain in the comments that no
> treatment is needed, and state the reason why.)
>
> Which part of code produces the message about missing e(b)?
> Apparently, you don't refer to it directly, but some of the code (in
> the -bootstrap-? in the -psmatch2-?) wants to get it. You would want
> to -set trace on- and may be -set tracedepth 3- or so to see who
> produces the message (increasing the tracedepth as needed if you can't
> find the culprit; without the tracedepth, you can get quite deeply
> into say the bootstrap code, with several nested levels of
> subroutines, and it will be difficult to tell where the problem really
> is).
>
> I have reservations about applicability of the bootstrap procedures
> with matching. I can buy a bootstrap procedure when the statistic is
> smooth. Matching, on the other hand, is intrinsically
> non-differentiable: when you take a new subsample, you will probably
> jump to a different neighbor. And jumps are bad. If somebody has a
> reference to a paper by say Imbens in say Journal of Econometrics
> where consistency of the bootstrap is established, I would be so-o-o
> relieved.
>
> On Wed, May 11, 2011 at 9:50 AM, Alistair Windsor
> <alistair.windsor@gmail.com> wrote:
>> Dear Statalisters,
>>
>> I am using a propensity score matching scheme to evaluate an education
>> intervention. Students self select into the intervention so some effort
>> needs to be taken to eliminate the selection bias in the data.
>>
>> I am aware of the Imbens result on the lack of asymptotic validity for
>> bootstrapping for matching schemes and I am trying some propensity score
>> reweighting schemes as well but thusfar the propensity score matching scheme
>> does the best job of eliminating observed difference and seems to be the
>> least sensitive to specification. In addition it is easy to explain.
>>
>> My problem concerns the fact that when I run my bootstrap command over my
>> matching scheme in a case where there is classification with no students in
>> it I get an error message
>>
>> e(b) not found
>>
>> and the bootstrap aborts. Altering the code so that it should adapt to empty
>> classifications has not helped. The relevant code in enclosed in a capture
>> block. It all ran fine until I turned it into a program and bootstrapped it.
>> Can anyone see my mistake?
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/