Re: st: how can i make my loop run faster?

[Partho Sarkar <partho.ss+lists@gmail.com>]

Hello Stefanlo

I am not quite sure I understand what you are proposing, but I suppose
this ought to be do-able,  though I can't offhand think of any direct
way to do this.  I think this would also be an extremely cumbersome
way to do this, and almost certainly  even slower than any of the
others we have talked about.  However, I had thought of another
possible approach:

Since you have a "short" panel- "only" 200 periods, but many more
firms, I would think dividing up your sample period into, say, 10
(non-overlapping) subsets, and doing a statsby regression for each
would give you the results for all (or selected subset of) firms
within each time sub-period.  You would still have to
combine(merge/append) the results.

But why not try the rolling loop as suggested in the thread I cited,
first splitting the firms into manageable subsets (which I think you
would have to do in any case, if you want to run this routine- unless
you have access to a super-computer!)  ?

By the way, I am curious to know what exactly the source of your data is!

Hope this helps

Partho


On Tue, Sep 20, 2011 at 10:20 AM, Stefano Rossi <sr525@cornell.edu> wrote:
> Dear Partho,
>
> many thanks for this, which is very useful.  I can see how "rolling" works, and I can see how it can generate efficiency gains, but I agree the whole procedure may still be quite slow and require splitting the sample into subsets to get a faster procedure in some way.
>
> I am currently considering a different path, namely generating a cross-section of observations by firm-period, whereby each firm-period unit contains 12 observations, from -1 to -12 (I would also have a separate data by +1 to +12).  This procedure would effectively produce a dataset 12 times larger than my current one.  This procedure would get around the "rolling" issue, and would allow me to use the "statsby" (or equivalent) command without worrying of the length of the estimation sample, with potentially large efficiency improvements (i.e., no "ifs").
>
> Provided my intuition is correct, my one concern here is how to create such dataset, which is 12 times bigger than the current one.  Is there a built-in Stata command that allows to do this efficiently?
>
> Many thanks for your support.
>
> Kind regards,
>
> Stefano
>
>
> ________________________________________
> From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] On Behalf Of Partho Sarkar [partho.ss+lists@gmail.com]
> Sent: Tuesday, September 20, 2011 12:33 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: how can i make my loop run faster?
>
> I guess Stefano might have solved his problem by now, but just to
> complete this, here is a post by  Brian R. Landy from an older thread
> which gives the complete code for -rolling-, including merging the
> results files.
>
> http://www.stata.com/statalist/archive/2009-09/msg01239.html
>
> The thread also points out the speed problems with rolling for panel data.
>
> P.Sarkar

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/