Re: st: Bootstrap error message

[Stas Kolenikov <skolenik@gmail.com>]

Alistair Windsor is bootstrapping some complicated results of a
propensity score matching procedure, and reports an obscure "e(b) not
found" message.

My reactions:

If you have a -capture-, you should have -if _rc{ }- following it with
treatment of the exception. Otherwise you just sweep the errors (that
you obviously are expecting to occur) under the carpet. (In my code, I
might leave an empty -else- and explain in the comments that no
treatment is needed, and state the reason why.)

Which part of code produces the message about missing e(b)?
Apparently, you don't refer to it directly, but some of the code (in
the -bootstrap-? in the -psmatch2-?) wants to get it. You would want
to -set trace on- and may be -set tracedepth 3- or so to see who
produces the message (increasing the tracedepth as needed if you can't
find the culprit; without the tracedepth, you can get quite deeply
into say the bootstrap code, with several nested levels of
subroutines, and it will be difficult to tell where the problem really
is).

I have reservations about applicability of the bootstrap procedures
with matching. I can buy a bootstrap procedure when the statistic is
smooth. Matching, on the other hand, is intrinsically
non-differentiable: when you take a new subsample, you will probably
jump to a different neighbor. And jumps are bad. If somebody has a
reference to a paper by say Imbens in say Journal of Econometrics
where consistency of the bootstrap is established, I would be so-o-o
relieved.

On Wed, May 11, 2011 at 9:50 AM, Alistair Windsor
<alistair.windsor@gmail.com> wrote:
> Dear Statalisters,
>
> I am using a propensity score matching scheme to evaluate an education
> intervention. Students self select into the intervention so some effort
> needs to be taken to eliminate the selection bias in the data.
>
> I am aware of the Imbens result on the lack of asymptotic validity for
> bootstrapping for matching schemes and I am trying some propensity score
> reweighting schemes as well but thusfar the propensity score matching scheme
> does the best job of eliminating observed difference and seems to be the
> least sensitive to specification. In addition it is easy to explain.
>
> My problem concerns the fact that when I run my bootstrap command over my
> matching scheme in a case where there is classification with no students in
> it I get an error message
>
> e(b) not found
>
> and the bootstrap aborts. Altering the code so that it should adapt to empty
> classifications has not helped. The relevant code in enclosed in a capture
> block. It all ran fine until I turned it into a program and bootstrapped it.
> Can anyone see my mistake?
>
> Many thanks in advance,
>
> Alistair
>
> My program is as follows:
>
> capture program drop difference
> program define difference, rclass
> local intervention  `1'
> local controls female i.race i.major freshgpa nfreshgpa
> local outcomes retained gpa
>
> qui: gen float weight = .
> foreach measure of local outcomes {
>  qui: gen att_`measure' = .
> }
>
> forvalues y = 1/2 {
>  qui: levelsof classification if `intervention'==1 & yr==`y',
> local(classifications)
>  foreach c of local classifications {
>    capture {
>      qui: xi: psmatch2 `intervention' `controls' ///
>        if yr  == `y' & classification == "`c'", ///
>          outcome(`outcomes') noreplacement descending common quietly
> nowarning
>      qui: replace weight = _weight if yr == `y' & classification == "`c'"
>      foreach measure of local outcomes {
>        qui: replace att_`measure' = r(att_`measure') if yr == `y' & ///
>          classification == "`c'" & `intervention' & weight == 1
>      }
>    }
>  }
> }
> sum att_gpa if `intervention' == 1 & weight == 1, meanonly
> return scalar gpa_diff = r(mean)
> sum att_retained if `intervention' == 1 & weight == 1, meanonly
> return scalar retained_diff = r(mean)
> qui: drop weight
> foreach measure of local outcomes {
>    qui: drop att_`measure'
>  }
>
> end
>
> The main body of the loop
>
> foreach intervention of local interventions {
>
>  log using ...
>
>  qui: drop if mstem & `intervention' == 0
>
>  bootstrap r(gpa_diff) r(retained_diff), seed(6784431) strata(`intervention'
> yr) reps(15) nodrop : difference `intervention'
>
>  log close
>
>  pause
>
>  restore, preserve // This restores the data set to where it was before I
> dropped students.
>
> }
>
> It took a while for me to discover the nodrop option. The code is a
> bastardized mix of Stata 10 and 11. Apologies.
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>



-- 
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/