 |
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
st: xtlogit is slow
On Sunday July 06, 2008 "Mitchell F. Berman" <[email protected]> reported a
speed issue for -xtlogit, pa-:
> I've noticed that xtlogit is slow compared to SAS genmod to get logistic
> regressions using an exchangeable correlation model. For large data sets
> (300,000 rows-15 variables) xtlogit takes 45 minutes, SAS takes 1.5 minutes.
> The results are identical.
> ...
The issue is actually associated with -xtgee-, which is used by -xtlogit- to
fit population-averaged models.
After analyzing Mitchell's data, we've determined that (as some contributors
on this thread have speculated) slow downs are not necessarily caused by large
datasets. The slow downs are caused by large panels -- Mitchell's data has
panels as large as 2,600 or so observations.
-xtgee- works through matrix manipulations involving working correlation
matrices at the panel level. In Mitchell's case -xtgee- is working with
square matrices of dimension as high as 2,600. -xtgee- in its current
implementation performs manipulations directly on these matrices, whereas an
implementation that takes into account the structure (and possible sparseness)
of these working correlations (e.g., exchangeable) would be much more
efficient.
As such, this is something that we will seriously look into adding.
-- Yulia Marchenko -- Bobby Gutierrez
[email protected] [email protected]
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/