RE: st: Successive rounds of a simulation become slower and slower

["Feiveson, Alan H. (JSC-SK311)" <alan.h.feiveson@nasa.gov>]

Michael - 

I had asked a similar question a couple of years ago. See if

http://www.stata.com/gsearch.php?q=increasing+time+burden&site=stata&client=stata&proxystylesheet=stata&output=xml_no_dtd&restrict=Statalist

and followup posts help you at all.


Al Feiveson

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Joerg Luedicke
Sent: Wednesday, October 26, 2011 12:50 PM
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: Successive rounds of a simulation become slower and slower

Hi Michael,

I don't think this can be answered without you showing your actual
code. If every simulation run really does the same thing, then the
time increase is linear. Therefore, something must be wrong with your
code and I suspect it has to do with your using several loops and the
stuff that is included/not included in each loop. But as I said, this
is not answerable without seeing what is actually going on.

Also, consider using Stata's -simulate-.

Joerg

On Wed, Oct 26, 2011 at 12:34 PM, Lacy,Michael
<Michael.Lacy@colostate.edu> wrote:
> Greetings,
>
> Successive rounds of a simulation I'm doing become progressively slower, so much so as to make continuing impractical. I'd like to figure out why this is happening
> and cure it. The first round might take 2 minutes, then 5 minutes for the next, then 15, etc. Each round *should* take the same amount of time, since the same estimation process
> is being performed on similar data at each round.
>
>
> Each round involves 1000 repetitions of simulating data based on several parameters. Statistics estimated at each repetition are -post-ed, and the temporary
> -postfile- is then saved to a permanent file. In pseucode outline, the overall structure  of the code is something like this:
>
> foreach p1 of ParamList1 {
>  foreach p2 of ParamList2 {
>    foreach p3 of ParamList3 {
>       Initialize for this round
>       forval r = 1/1000 {  // repetitions
>         Set up for posting
>         Create simulated data based on p1 p2 p3
>         Run a user-written estimation process that uses several Mata programs
>         Post the current estimates
>       }
>       Save postfile to a permanent file
>       Erase postfile
>    }
>  }
>
>
> There are several parameters in each list, so that there are some 100+ rounds to be run, which should take a few hours in total.  But because of the round to round decrement
> in performance, the overall simulation won't finish in 10-12 hours.  I'm running 11.2 SE, under Win XP, on a modest (2G RAM) machine.  There should not be any
> memory issues, since the data created for each round involves something like 10  variables for 10-20 observations; similarly, the postfile for each round is nothing big,
> perhaps 10 variables for 1000 cases. I also seem to be experiencing the same problem running 11.2 SE on a Win 7 machine, with 8G.
>
> Regards,
>
> Mike Lacy
> Colorado State University
> Fort Collins CO 80523-1784
> 970.491.6721 (voice)
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/