st: Nonparametric multilevel residuals bootstrapping in Stata?

[<S.Jenkins@lse.ac.uk>]

------------------------------

Date: Thu, 18 Jul 2013 16:32:30 -0400
From: Jordan Silberman <silberman.stata@gmail.com>
Subject: st: Nonparametric multilevel residuals bootstrapping in Stata?

Hi,

I'd like to bootstrap a multilevel model using nonparametric residuals
bootstrapping (aka "semiparametric bootstrapping"). The idea is that
you compute the multilevel model using xtmixed or something similar
and compute residuals at each level of the model. For each bootstrap
iteration you compute new values of the DV by multiplying values of
fixed effect predictors by the appropriate coefficients and then
adding a residual from each level. These residuals are randomly
sampled (with replacement) from the observed residuals. Then you run
the model with the resultant values of the outcome variable specified
as the DV.

The problem with this approach is that the residuals computed by the
multilevel model are shrunk toward unit-specific means--they
underestimate the magnitude of the residuals you'd expect to observe
in a new sample from the population. So, the residuals must be
"unshrunk." In the article pasted below, Carpenter et al. (2003)
present a method for doing this that "unshrinks" the residuals and
also reproduces the appropriate covariance structure of the residuals.

Does anyone know if there's a relatively simple way of reproducing
this procedure in Stata? MLwiN can do it, but I'd like to be able to
do it in Stata.

Thanks!
Jordan

Carpenter, J.R., Goldstein, H., and Rasbah, J. (2003), “A novel
bootstrap procedure for assessing the relationship between class size
and achievement.” Appl. Statist. , Vol. 52, pp 431-443

=================================================

Have you checked out -runmlwin- on SSC?  It's a terrific program for running MLwiN via Stata, and getting output back in Stata's saved results. 

The 'reflated' option under residuals options allows creation of unshrunken residuals.

When I looked into implementing the Carpenter et al. procedure, I recall that one still has to write a bootstrapping wrapper. Put another way, I recall that virtually all of the functionality in MLwiN can be derived directly via -runmlwin- ... with the notable exception of this bootstrapping method. (I never had the time to implement the method.)

I would also point out that the Carpenter et al. procedure is no panacea. If you look at Table 1 in the article cited (Monte-Carlo simulation results), you see that their method improves coverage rates relative to the conventional 'parametric' reference point. However, nominal coverage rates can still be way way below the 90% reference point.


Stephen
------------------
Stephen P. Jenkins <s.jenkins@lse.ac.uk>

Please access the attached hyperlink for an important electronic communications disclaimer: http://lse.ac.uk/emailDisclaimer

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/