Re: st: RE: Using macros and batch running.

[Austin Nichols <austinnichols@gmail.com>]

Adrian Sayers <adrian.sayers@bristol.ac.uk> :
Use an ado file that defines a -program- to do everything you are
currently -display-ing to a log file below. The -program- creates
data, does estimation etc., then reports results as scalars.  Then run
-simulate- to call that program.  If the contents of the program are
not well parallelized, then you can benefit by calling multiple
instances of Stata because Stata's parallelization is small-grain, not
large-grain.  You need to calculate the seed to start from, though.
Let's say you calculate that the -program- uses no more than 3000
random numbers (you need to check this carefully, as many programs you
use may generate random numbers).  Then you can run 1000 simulations
using 10 instances of Stata each running 100 simulations by writing a
do file that has three lines to just run -simulate- and save the
results, then writing another do file that calls the ancillary do file
10 times (also incrementing the seed 3000 times, or as required). If
there are outputs from one simulation that affect a later simulation,
then you really do have a serial process and this will not work, but
the example code you gave does not illustrate such a case.

-----begin ado file myprog.ado--------
prog myprog, rclass
syntax [anything] [, opt(string) ]
* machinery here
return scalar result=r(mean)
eret clear
end
-----end do file--------

-----begin ancillary do file mysim.do--------
args seed id
simulate, r(100) seed(`seed'): myprog x y z, opt(whatever)
save run`id'
-----end do file--------

--------begin main do file---------
set seed 1
forv i=1/10 {
 winexec "\\mypath\stata11\StataMP.exe" -b do mysim `c(seed)' `i'
 forv r=1/3000 {
  loc u=uniform()
  }
}
---------end main do file----------

On Mon, Jul 4, 2011 at 7:17 AM, Adrian Sayers
<adrian.sayers@bristol.ac.uk> wrote:
> Hi Daniel,
>
> I had thought of righting a header file. (i.e. .doh) but i am not sure this
> will solve my problem. I should probably explain what i am trying to do
> better.
>
> Currently i am running series of simulations, which take 3+ days to complete
> on my PC.
>
> Most of the process are serial, therefore their is limited gains in
> parrelisation. However i had hoped that i could invoke stata in numerous
> windows, this would allow me to perform all experiments in the simulation
> simultaneously, drastically cutting the running time from 3 days to 20
> minutes.
>
> To do this i need to modify do files on the fly, so to speak. Typically if
> you were running this on only 1 machine i would bundle this in a loop and
> just wait for it to run serially.
>
> My second attempt is to write a do file, which in turns writes all my other
> do files.
>
> But this is not pretty, or very elegant.
> Hopefully this will give you a better idea of what i am trying to achieve.
>
> Cheers
> Adrian
>
> E.g.
> ****************************************************************************
> **
> * Create the do files in code... Basically Prefixing all code with display
> foreach i in 1  2 3 {
> cap log close
> qui log using test`i'.do  , text replace
> di "clear all                         "
> di "set obs 1000                      "
> di "gen id = _n                       "
> di "gen x = `i'*uniform()             "
> di "save testdata`i' , replace        "
>
>
> qui log close
> }
>
> * execute batch created do files using the batch function.
> forvalues i = 1/3 {
> winexec "\\mypath\stata11\StataMP.exe" -e include test`i'.do
> }
>
> ****************************************************************************

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/