Re: st: RE: Run time for random effects Poisson model

[Owen Gallupe <ogallupe@gmail.com>]

Thanks for your reply, David. I tried playing around with some of the
more "problematic" variables but the situation didn't improve. My next
approach was to run a series of models adding a single random effect
each time. The random intercept only model took maybe 30 seconds;
adding 1 random coefficient increased the time to about 4 minutes; two
random coefs took about 24 minutes; three random coefs took about 145
minutes; four random coefs is currently at over 21 hours and counting.
So adding random effects appears to increase the run time
exponentially. I think I will go with a fixed effects approach as this
run time can't be sustained for an analysis that will require multiple
runs with a variety of outcomes and subsamples. Also, a fixed effects
approach nicely fits my interests in this case.

Though rumour has it that Stata 13 is faster. I should also probably
upgrade SE to MP.


On Wed, Sep 18, 2013 at 3:53 PM, Jacobs, David
<jacobs.184@sociology.osu.edu> wrote:
> In case no one replied, I think something's wrong because your runs should not take that long.
>
> I'd start by finding out if some of your explanatory variables differ by huge amounts.  Count estimators don't seem to handle a situation in which some variables are proportions constrained between zero and one, while other explanatory variables are like the population of cities and vary between say 50,000 and 8,000,000.  Huge differences like this stop count models from converging.
>
> Dave Jacobs
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Owen Gallupe
> Sent: Tuesday, September 17, 2013 7:42 PM
> To: statalist@hsphsun2.harvard.edu
> Subject: st: Run time for random effects Poisson model
>
> Hi,
>
> I'm hoping someone could give me an educated guess on how long it would take to run random effects Poisson model with 7 random coefficients.
>
> There are about 2500 cases and 20 IVs at level 1 (more details at bottom).
>
> I already aborted one attempt after about an hour when I figured something must have gone wrong. But I did some research and I was clearly just being impatient. However, now I am nearly 15 hours in and it still appears to be chugging along.
>
> Is this likely to take a few more hours? Days? Weeks? I can handle a couple days, but I will have to rethink my approach if it is likely to go beyond that as I have a series of models that need to be run.
>
> Thanks.
>
> Owen
>
> The syntax is:
> xtmepoisson dv iv1 iv2 iv3 iv4 iv5 iv6 iv7 iv8 iv9 iv10 iv11 iv12 iv13 iv14
> iv15 iv16 iv17 iv18 iv19 iv20 || group: iv4 iv5 iv6 iv7 iv8 iv9 iv10,
> cov(unstructured)
>
> I am using Stata SE 12.1 (64 bit) with Windows 7 on a computer with an Intel i7-2640M (2.40 GHz) and 8GB RAM.
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/