Re: st: Bootstrap error message

[Stas Kolenikov <skolenik@gmail.com>]

OK. I would love to see a reference to Imbens showing inconsistency,
either way will do. Do you have the reference in your book?

On Wed, May 11, 2011 at 11:13 AM, Austin Nichols
<austinnichols@gmail.com> wrote:
> Stas--
> Alistair says he is aware of the result from Imbens that shows the
> bootstrap is inconsistent for matching, so quite the opposite of what
> you wanted him to cite.  Why anyone would pursue the bootstrap knowing
> it will not produce correct SEs is beyond me.
> He also claims that reweighting fails to achieve balance, which I find
> implausible at best.
>
> On Wed, May 11, 2011 at 11:21 AM, Stas Kolenikov <skolenik@gmail.com> wrote:
>> Alistair Windsor is bootstrapping some complicated results of a
>> propensity score matching procedure, and reports an obscure "e(b) not
>> found" message.
>>
>> My reactions:
>>
>> If you have a -capture-, you should have -if _rc{ }- following it with
>> treatment of the exception. Otherwise you just sweep the errors (that
>> you obviously are expecting to occur) under the carpet. (In my code, I
>> might leave an empty -else- and explain in the comments that no
>> treatment is needed, and state the reason why.)
>>
>> Which part of code produces the message about missing e(b)?
>> Apparently, you don't refer to it directly, but some of the code (in
>> the -bootstrap-? in the -psmatch2-?) wants to get it. You would want
>> to -set trace on- and may be -set tracedepth 3- or so to see who
>> produces the message (increasing the tracedepth as needed if you can't
>> find the culprit; without the tracedepth, you can get quite deeply
>> into say the bootstrap code, with several nested levels of
>> subroutines, and it will be difficult to tell where the problem really
>> is).
>>
>> I have reservations about applicability of the bootstrap procedures
>> with matching. I can buy a bootstrap procedure when the statistic is
>> smooth. Matching, on the other hand, is intrinsically
>> non-differentiable: when you take a new subsample, you will probably
>> jump to a different neighbor. And jumps are bad. If somebody has a
>> reference to a paper by say Imbens in say Journal of Econometrics
>> where consistency of the bootstrap is established, I would be so-o-o
>> relieved.
>>
>> On Wed, May 11, 2011 at 9:50 AM, Alistair Windsor
>> <alistair.windsor@gmail.com> wrote:
>>> Dear Statalisters,
>>>
>>> I am using a propensity score matching scheme to evaluate an education
>>> intervention. Students self select into the intervention so some effort
>>> needs to be taken to eliminate the selection bias in the data.
>>>
>>> I am aware of the Imbens result on the lack of asymptotic validity for
>>> bootstrapping for matching schemes and I am trying some propensity score
>>> reweighting schemes as well but thusfar the propensity score matching scheme
>>> does the best job of eliminating observed difference and seems to be the
>>> least sensitive to specification. In addition it is easy to explain.
>>>
>>> My problem concerns the fact that when I run my bootstrap command over my
>>> matching scheme in a case where there is classification with no students in
>>> it I get an error message
>>>
>>> e(b) not found
>>>
>>> and the bootstrap aborts. Altering the code so that it should adapt to empty
>>> classifications has not helped. The relevant code in enclosed in a capture
>>> block. It all ran fine until I turned it into a program and bootstrapped it.
>>> Can anyone see my mistake?
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>



-- 
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/