Re: st: STATA 12 very slow compared to STATA 10

[Nick Cox <njcoxstata@gmail.com>]

In addition to other comments, my own are contradictory to some
extent. Allow me to try to summarise:

If your problem obliges you to use matrices, then Mata is to be preferred.

However, what you showed us of your program did not convey to me the
impression that you need to use matrices.

Nick

On Tue, Mar 12, 2013 at 4:46 PM, Sara Borelli <saraborelli77@gmail.com> wrote:
> Nick,
>
> thank you very much for your suggestions
>
> I did not use Mata before, I tried to play around with it but it seems
>  it cannot be combined with loop...
> Since I have several loops in my program, would you have an example or
> suggestion of how I could implement some of my program using Mata and
> in loop?
>
> Thanks for any input you might have
>
> Sara
>
>
> On Tue, Mar 12, 2013 at 10:29 AM, Nick Cox <njcoxstata@gmail.com> wrote:
>> Maarten [NB] understated the difficulty for everyone but you, as the
>> speed of Stata [NB] can depend on what else is running, how much
>> memory is available, etc., etc.
>>
>> A quite different comment is what are you doing here and why? I see
>> lots of pushing variables in and out of Stata matrices and some
>> bootstrap sampling, so four points as you are worried about speed.
>>
>> 0. I see no point in using Stata matrices for storing data unless you
>> want to do matrix manipulations.
>>
>> 1. Mata matrices are faster than Stata matrices. That was true in Stata 10 too.
>>
>> 2. When bootstrapping, it seems easiest to hold each sample in memory
>> once, process it and then throw it away.
>>
>> 3. -mmerge- [source not given, but last updated on SSC in 2002?]  is
>> long since superseded by a revised -merge-.
>>
>> Naturally I can't comment on the rest of the program.
>>
>> Nick
>>
>> On Tue, Mar 12, 2013 at 1:40 PM, Sara Borelli <saraborelli77@gmail.com> wrote:
>>> Hi Martin
>>> you are right, I did not ask the question correctly. I thought there
>>> was maybe a common problem for STATA 12 being slow in general that
>>> others might have encountered. I apologize for that
>>>
>>> I write here below an extract of the program I am running. If you have
>>> any insight of what is going wrong I would really appreciate it. I
>>> also tried to run the entire program using "version 10" at the
>>> beginning of the do file, but it is still very slow
>>> I let the program run all night but is still not done. When I was
>>> using STATA 10 it was taking only 15 minutes overall
>>>
>>>
>>> before this "mat define" command line I am running some glm estimation
>>> that works at usual speed
>>>
>>> mat define P=J(618,1,.)
>>>
>>> set seed 55982264
>>>
>>> forvalues x=1/500 {
>>>
>>> mat define BS_r`x'=J(618,1,.)
>>>
>>> use data1
>>> for each var of varlist v1 v2 v3 v4 ....v10 {
>>> bsample
>>> mkmat `var', mat(Z_`var'_r`x')
>>> mat BS_r`x'=BS_r`x',Z_`var'_r`x'
>>> }
>>> drop v1-v10
>>> svmat BS_r`x', names(col)
>>> gen id=_n
>>> save BS_r`x', replace
>>>
>>> use data2, clear
>>> gen id=_n
>>> mmerge id using BS_r`x'
>>> save replication `x'
>>> predict prediction`x'
>>> mkmat prediction`x', mat(P_`x')
>>> mat P=P, P_`x'
>>> }
>>>
>>> It seem to run smoothly until the "predict", but seems to get slow at
>>> the time of running the command  mkmat prediction`x', mat(P_`x')
>>>
>>> thanks for any insight you might have
>>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/