Re: st: xtlogit is slow

[David Airey <david.airey@Vanderbilt.Edu>]

.

It takes 54 seconds on my laptop.

-Dave

. do "/var/folders/AJ/AJjttUSSFdSi1HiInJZyY++++TI/-Tmp-//SD00651.000000"

. clear *

. set more off

. set memory 100M

Current memory allocation

                    current                                 memory  
usage
    settable          value     description                 (1M =  
1024k)
     
--------------------------------------------------------------------
    set maxvar         5000     max. variables allowed           2.067M
    set memory          100M    max. data space                100.000M
    set matsize         400     max. RHS vars in models          1.254M
                                                            -----------
                                                               103.321M

. set seed `=date("2008-07-08", "YMD")'

. set obs 300000
obs was 0, now 300000

. *
. generate byte depvar = uniform() > 0.5

. generate byte catvar1 = floor(4 * uniform())

. forvalue i = 1/12 {
  2. generate float other_var`i' = uniform()
  3. }

. generate byte rand = floor(9 * uniform())

. generate int panelvar = floor(3000 * uniform())

. timer clear 1

. timer on 1

. xi: xtlogit depvar i.catvar1 other_var* if rand <7, i(panelvar) pa
i.catvar1         _Icatvar1_0-3       (naturally coded; _Icatvar1_0  
omitted)

Iteration 1: tolerance = 2.886e-06
Iteration 2: tolerance = 2.193e-10

GEE population-averaged model                   Number of obs       
=    233408
Group variable:                   panelvar      Number of groups    
=      3000
Link:                                logit      Obs per group: min  
=        47
Family:                           binomial                     avg  
=      77.8
Correlation:                  exchangeable                     max  
=       106
                                                Wald chi2(15)       
=     18.77
Scale parameter:                         1      Prob > chi2         
=    0.2245

------------------------------------------------------------------------------
      depvar |      Coef.   Std. Err.      z    P>|z|     [95% Conf.  
Interval]
------------- 
+----------------------------------------------------------------
 _Icatvar1_1 |   .0077015   .0116875     0.66   0.510    -. 
0152056    .0306086
 _Icatvar1_2 |   .0213159   .0117154     1.82   0.069    -. 
0016459    .0442778
 _Icatvar1_3 |  -.0017835   .0117167    -0.15   0.879    -. 
0247479    .0211809
  other_var1 |   .0003397   .0143455     0.02   0.981    -. 
0277769    .0284563
  other_var2 |  -.0074464   .0143168    -0.52   0.603    -. 
0355067    .0206139
  other_var3 |  -.0054181   .0143215    -0.38   0.705    -. 
0334876    .0226514
  other_var4 |   .0065459   .0143327     0.46   0.648    -. 
0215456    .0346375
  other_var5 |   .0086643   .0143549     0.60   0.546    -. 
0194707    .0367994
  other_var6 |    .024404   .0143424     1.70   0.089    -. 
0037066    .0525146
  other_var7 |   .0125822   .0143138     0.88   0.379    -. 
0154724    .0406369
  other_var8 |  -.0029192   .0143368    -0.20   0.839    -. 
0310189    .0251804
  other_var9 |    -.00545   .0143595    -0.38   0.704    -. 
0335942    .0226941
 other_var10 |   .0050038   .0143494     0.35   0.727    -. 
0231206    .0331282
 other_var11 |  -.0380009   .0143391    -2.65   0.008     -.066105    
-.0098967
 other_var12 |   .0203525   .0143351     1.42   0.156    -. 
0077437    .0484488
       _cons |  -.0186582   .0262054    -0.71   0.476    -. 
0700199    .0327035
------------------------------------------------------------------------------

. timer off 1

. timer list
   1:     54.86 /        1 =      54.8610

. exit

end of do-file



On Jul 8, 2008, at 7:56 AM, Joseph Coveney wrote:


> Mitchell F. Berman wrote:
>
> Good thought on the xtlogit issue, but no, the if clause is not the
> problem.  Takes closer to an hour, I think, for the entire data set,  
> but
> I've only done this once.  I can tell that it is running slowly,  
> because
> each iteration takes many minutes.
> I sent this issue in to Stata Tech Support.  I'm hoping they  
> investigate it.
>
> --------------------------------------------------------------------------------
>
> I'm curious to see what they discover, because the do file below,  
> which has
> 15 variables, 300,000 observations, and the command line that you  
> showed
> earlier in the thread takes only 2.5 minutes (149.9250 seconds) on my
> boat-anchor-disguised-as-a-PC.  There must be something very  
> peculiar about
> your particular dataset or machine set up.
>
> Joseph Coveney
>
> clear *
> set more off
> set memory 100M
> set seed `=date("2008-07-08", "YMD")'
> set obs 300000
> *
> generate byte depvar = uniform() > 0.5
> generate byte catvar1 = floor(4 * uniform())
> forvalue i = 1/12 {
> generate float other_var`i' = uniform()
> }
> generate byte rand = floor(9 * uniform())
> generate int panelvar = floor(3000 * uniform())
> timer clear 1
> timer on 1
> xi: xtlogit depvar i.catvar1 other_var* if rand <7, i(panelvar) pa
> timer off 1
> timer list
> exit
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/