RE: st: MP running no faster than IC

[Timothy Mak <tshmak@hku.hk>]

Hi Ted, 

I do sympathize with you, cos I did buy a Stata MP2 at some point, thinking it might be able to speed up my mata codes. It probably didn't a whole lot, if at all (I was using Stata 10). 

Tim

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Ted Player
Sent: 09 July 2013 22:57
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: MP running no faster than IC

Why would I want a 2-second estimation process to be shortened to a .1
estimation process?  Because some of my work is with Monte Carlo
simulations that take days to run.  I am disappointed to find they are
not quicker at all with MP.

Based on the comments here and other reading I've done, it looks like
the parallelization procedures that Stata uses don't help anyone
except users who run really complex models on huge datasets.  It would
have been nice if Stata had said that up front (e.g., in their sales
materials alongside their glowing claims of increased speed for
everyone) rather than mentioning it parenthetically in obscure blog
posts.

Given Stata's sales pitch, I'm forced to wonder how many people have
taken the biggest hit they could afford from their research budget to
buy MP-2 core rather than SE, run it without any comparison to SE (or
set processors 1), and never have any idea that the extra money that
they spent isn't giving them any real benefit whatsoever. It's a pity.

On 7/8/13, Lucas <lucaselastic@gmail.com> wrote:
> Stata isn't over-claiming.  They just probably never though that
> someone running a command that takes 2 seconds would be seeking to run
> it even faster.  My jobs, and jobs of other people I know, routinely
> run days or weeks.  (And, yes, it is identified, everything checks
> out, it is just the data is massive and the model appropriately
> complex).  It is for such jobs that one needs parallel processing.
> Running the same 2 second command 500 times can't be parallelized with
> any efficiency because the overhead of managing the allocation of
> tasks swamps any gains attributable to parallelization.  Stata's only
> fault--if fault it be--is not making clear that unless one uses big
> data or finds oneself in situations that one model takes days or
> weeks, MP is of dubious value.  But, on the other hand, users seeking
> to run a regression model in .1 second rather than 2 seconds only
> inspire one to ask, "Why?"
>
> On Mon, Jul 8, 2013 at 9:09 PM, Ted Player <ted.player.660@gmail.com>
> wrote:
>> The benchmark tests I originally described were conducted on a local
>> machine.  I did a follow-up with an EC2 machine (as described
>> elsewhere in this thread).
>>
>> I see now that buried on p. 231 of Stata's MP performance report is
>> the mention that to get the improvements that Stata claims for
>> regression requires a single regression model with 180 regressors and
>> a dataset with 1,500,000 observations.  I usually do things like
>> bootstrap analyses on datasets with 500 observations, so I guess MP
>> isn't any more useful to me than SE.
>>
>> It looks like I fell for the advertising hype on
>> http://www.stata.com/statamp .  It's my fault for thinking Stata
>> wouldn't overclaim to make their software seem better than it really
>> is.  Live and learn I guess!
>>
>>
>>
>> On Mon, Jul 8, 2013 at 9:42 PM, Sergiy Radyakin <serjradyakin@gmail.com>
>> wrote:
>>> Dear Ted, I've witnessed many times that MP works much faster the IC.
>>> The figures in the report do make sense. No looking at your example:
>>> the only parallelizable part here is the "regress mpg weight gear
>>> foreign." Two things to notice immediately are the following:
>>>
>>> 1) the dataset contains 74 observations. The overhead of parallelizing
>>> it into 12 CPUs or even 4 CPUs is large relative to the size of the
>>> task at hand. You are likely to see the benefits of parallelization
>>> when you -expand- your dataset, say 1000000 (10^6) times and perhaps
>>> reduce the number of bootstrap iterations.
>>>
>>> 2) the dataset contains 74 observations. So the _regress command
>>> (internal) takes, say, 0.00001second and with parallelization takes
>>> may be 0.000001 second, but then you have 2 seconds of writing the
>>> output to the screen and scrolling the output window.  That is not
>>> parallelized (correct me if I am wrong), though scrolling seems to
>>> work much faster in recent versions (THANKS!) So, try disabling the
>>> output with -quietly- and you will see more performance gain from MP.
>>>
>>> 3) finally, Stata's ado files seem to not be parallelizable (you don't
>>> write them that way), but only internal commands are. There have been
>>> some changes in the most recent versions and the idea is to permit the
>>> users to write parallel code. I am yet to see these facilities, but it
>>> makes no sense to test parallelization benefits on do/ado code or
>>> where such code executes for a significant amount of time. This is
>>> also a reason while there is no need to separately benchmark bootstrap
>>> commands.
>>>
>>> To summarize the above, try the following commands on LARGE datasets
>>> (occupy e.g. half of your memory with data):
>>> mlogit - you should see performance increase about 3 times on a 12
>>> CPUs MP vs 1CPU IC.
>>> summarize - you should see about 11-fold performance increase on a
>>> 12CPUs MP vs 1CPU IC
>>>
>>> Run tests on a local machine. Perhaps it's the Amazon that is to blame
>>> (I don't mean it). Some hosters limit your TOTAL computing power, so
>>> you can get 128 cores with the same total performance as 1 core. Then
>>> you are better of with a single CPU license of course :)
>>>
>>> Hope this helps.
>>> Best, Sergiy Radyakin
>>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
>
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/