Re: st: AW: Bootsrapping standard errors of elasticities

[Stas Kolenikov <skolenik@gmail.com>]

On Fri, Mar 5, 2010 at 7:46 AM, Martin Weiss <martin.weiss1@gmx.de> wrote:

> It`s been tried before:
> http://www.stata.com/statalist/archive/2008-02/msg00525.html
>
> In Stata 11, -margins- is responsible for the calculation of elasticities,
> and there is no -bootstrap-ped standard error for it.
>

In large samples, there should be little to no difference between the
bootstrap and the robust standard errors that -margins- will generate. Hence
I personally see little point in complicating things to the degree of using
-bootstrap-. If anything, I would view the (dis)similarities between the
robust and the bootstrap standard error as a measure of the model (mis)fit.
And if the standard errors are indeed notably different (by say 50% or
more), then I would not trust either of them, unfortunately.

Note that in some situations figuring out a good bootstrap scheme is a
challenge in and of itself. In the example that Martin posted elsewhere, he
showed a way to bootstrap a linear regression model. Now, I am aware of at
least three different bootstrap schemes for regression analysis, and the
choice between them is a matter of taste (and the degree of programming
skill):

1. bootstrap the pairs (x, y) as Martin did -- very little work;

2. obtain the residuals from the main regression, keep all the explanatory
variables at their respective sample values and bootstrap the residuals --
requires extra coding and adding the residuals back to the linear
prediction. Implicitly assumes the residuals to be i.i.d. (hardly true with
auto data on prices), and the data generating model to be correctly
specified (hardly true with auto data, either). But doing the bootstrap
conditional on the explanatory variables makes far more sense in
applications like designed experiments in which you try pretty hard to
obtain a balanced design with orthogonal factors.

3. implement the wild bootstrap, in which you keep all the explanatory
variables at their respective sample values, and create an auxiliary
distribution for each sample point with certain variance and skewness
properties. Usually this auxiliary distribution is a two-point mass
distribution centered at zero, with variance = residual^2 (or residual^2
divided by the hat value) and skewed left or right depending on the sign of
the residual. (I've never seen it implemented in Stata, although of course I
cannot claim full coverage of all econometric and applied microeconomics
journals.)

Sometimes, there are issues with scaling, as somebody warned on statalist
regarding the bootstrap of factor analysis. Similar issues may arise with
limited dependent variable models that rely on arbitrary scaling
(Var[residuals]=a fixed constant). Also, some of the bootstrap samples may
give perfect prediction for logit/probit models, or non-invertible sample
covariance matrices in factor analysis (X'X matrices in regression
analysis).

If you have complex survey data, or clustered data, or time-series data, or
some other non-independent data, that adds another level of complication to
any of the bootstrap schemes.

As a bottom line, the more I learn about the bootstrap, the less trust I
have regarding any specific application of it.

-- 
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/