Re: st: Problem with seed and bootstrap

[Gary Longton <glongton@fhcrc.org>]

Alejandro wrote:

> I have a problem with a program that I wrote. For some
> reason I can't replicate my standard errors results when
> using bootstrap. I guess something is missing in the ado
> that I wrote but I can't figure what it is. The program is
> defined as eclass and the matrices b and V seem to be
> working. 
> If I run the following do file 
> set seed 345478595
> bs "myprogram x y z" _b
> set seed 345478595
> bs "myprogram x y z" _b
>
> I get different standard errors each time. The same happens
> when I use the syntax for Stata 9 (altough for some reason
> it doesn't like the _b...).
>
> bs x y z, seed(345478595):myprogram x y z
> bs x y z, seed(345478595):myprogram x y z
>
> Any ideas?
Alejandro,

An idea, though it may not be the problem in your particular program.

I have, on a couple of occasions, run into a reproducibility problem 
running simulations (given identical starting seed as in your case) - 
though I would expect that the same problem might occur with -bootsrap- 
under otherwise similar circumstances.

The problem has occurred in situations where the (1) calculation of 
results in the program, for a given sample, depend in some way on sort 
order, AND (2) when there is a -sort- or -bysort- in the program on a 
variable or combination of variables that doesn't uniquely distinguish 
observations.  I.e., apparently there is some variability in the result 
of a -sort- on the same dataset when the values of the sort variables 
don't completely distinguish individual observations.

I've gotten around this by defining a unique observation identifier and 
including it as the last variable in the sort.  I suspect this may also 
be the purpose of the -stable- option to -sort-, though I have not tried 
this as a solution in the situations where it has been a problem (I did 
not notice the -stable- sort option until sometime after dealing with 
the reproducibility problem).

- Gary


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/