Re: st: MP running no faster than IC

[Lucas <lucaselastic@gmail.com>]

I guess stata assumed statistical analysts who use computational
methods would know the basic constraints of parallelization.  Indeed,
as multi-tasking is a common activity these days, the basic
principle--the task must be large enough that the speed gains
overwhelm the management costs--is so obvious that it makes sense they
may not mention it. I mean, analogically, who simultaneously
handwashes their dinner dishes while parking their car in the garage?
Though one could do them simultaneously, it is clear that the
management costs (putting down the soapy dish, drying your hands,
walking from the sink to the location of the car, opening the car door
(if it is not already open), getting into the driver's seat, . . .)
are so much higher than any possible savings, it is better to simply
do one task, then do the other.  Multi-cores do not extricate one from
this basic efficiency challenge.

On the other hand, I also have done monte carlo work, so I sympathize
with the frustration of the long wait. If you are equally frustrated,
you may not want to dump the mp version just yet.  Was the task in
your test similar to the tasks you really plan to do.  It is likely
that if you had a more complex regression model that took 20 seconds,
say, the speed gains might kick in enough to overwhelm management
(and, I say, overwhelming management is always a good thing :-)  ).
So, you may want to run your test on something larger, more akin to
what you really plan to analyze, before you ask for a refund for the
MP version.

Finally, I really do sympathize with your plight.  When stata-mp was
first introduced, I was really looking forward to using it.  Then I
read their full documentation and found the dreaded words conveying
that the xtmixed commands had not been parallelized.
AAAARRRRGGGGHHHHHH!!!!!!

My frustration notwithstanding, this only goes to show that it is
always useful to read the documentation on your key issues.

And, fortunately for me and many others, the mixed suite has now been
brought under the parallelization regime.

Best of luck with your research.

Respectfully,
Sam


On Tue, Jul 9, 2013 at 7:56 AM, Ted Player <ted.player.660@gmail.com> wrote:
> Why would I want a 2-second estimation process to be shortened to a .1
> estimation process?  Because some of my work is with Monte Carlo
> simulations that take days to run.  I am disappointed to find they are
> not quicker at all with MP.
>
> Based on the comments here and other reading I've done, it looks like
> the parallelization procedures that Stata uses don't help anyone
> except users who run really complex models on huge datasets.  It would
> have been nice if Stata had said that up front (e.g., in their sales
> materials alongside their glowing claims of increased speed for
> everyone) rather than mentioning it parenthetically in obscure blog
> posts.
>
> Given Stata's sales pitch, I'm forced to wonder how many people have
> taken the biggest hit they could afford from their research budget to
> buy MP-2 core rather than SE, run it without any comparison to SE (or
> set processors 1), and never have any idea that the extra money that
> they spent isn't giving them any real benefit whatsoever. It's a pity.
>
> On 7/8/13, Lucas <lucaselastic@gmail.com> wrote:
>> Stata isn't over-claiming.  They just probably never though that
>> someone running a command that takes 2 seconds would be seeking to run
>> it even faster.  My jobs, and jobs of other people I know, routinely
>> run days or weeks.  (And, yes, it is identified, everything checks
>> out, it is just the data is massive and the model appropriately
>> complex).  It is for such jobs that one needs parallel processing.
>> Running the same 2 second command 500 times can't be parallelized with
>> any efficiency because the overhead of managing the allocation of
>> tasks swamps any gains attributable to parallelization.  Stata's only
>> fault--if fault it be--is not making clear that unless one uses big
>> data or finds oneself in situations that one model takes days or
>> weeks, MP is of dubious value.  But, on the other hand, users seeking
>> to run a regression model in .1 second rather than 2 seconds only
>> inspire one to ask, "Why?"
>>
>> On Mon, Jul 8, 2013 at 9:09 PM, Ted Player <ted.player.660@gmail.com>
>> wrote:
>>> The benchmark tests I originally described were conducted on a local
>>> machine.  I did a follow-up with an EC2 machine (as described
>>> elsewhere in this thread).
>>>
>>> I see now that buried on p. 231 of Stata's MP performance report is
>>> the mention that to get the improvements that Stata claims for
>>> regression requires a single regression model with 180 regressors and
>>> a dataset with 1,500,000 observations.  I usually do things like
>>> bootstrap analyses on datasets with 500 observations, so I guess MP
>>> isn't any more useful to me than SE.
>>>
>>> It looks like I fell for the advertising hype on
>>> http://www.stata.com/statamp .  It's my fault for thinking Stata
>>> wouldn't overclaim to make their software seem better than it really
>>> is.  Live and learn I guess!
>>>
>>>
>>>
>>> On Mon, Jul 8, 2013 at 9:42 PM, Sergiy Radyakin <serjradyakin@gmail.com>
>>> wrote:
>>>> Dear Ted, I've witnessed many times that MP works much faster the IC.
>>>> The figures in the report do make sense. No looking at your example:
>>>> the only parallelizable part here is the "regress mpg weight gear
>>>> foreign." Two things to notice immediately are the following:
>>>>
>>>> 1) the dataset contains 74 observations. The overhead of parallelizing
>>>> it into 12 CPUs or even 4 CPUs is large relative to the size of the
>>>> task at hand. You are likely to see the benefits of parallelization
>>>> when you -expand- your dataset, say 1000000 (10^6) times and perhaps
>>>> reduce the number of bootstrap iterations.
>>>>
>>>> 2) the dataset contains 74 observations. So the _regress command
>>>> (internal) takes, say, 0.00001second and with parallelization takes
>>>> may be 0.000001 second, but then you have 2 seconds of writing the
>>>> output to the screen and scrolling the output window.  That is not
>>>> parallelized (correct me if I am wrong), though scrolling seems to
>>>> work much faster in recent versions (THANKS!) So, try disabling the
>>>> output with -quietly- and you will see more performance gain from MP.
>>>>
>>>> 3) finally, Stata's ado files seem to not be parallelizable (you don't
>>>> write them that way), but only internal commands are. There have been
>>>> some changes in the most recent versions and the idea is to permit the
>>>> users to write parallel code. I am yet to see these facilities, but it
>>>> makes no sense to test parallelization benefits on do/ado code or
>>>> where such code executes for a significant amount of time. This is
>>>> also a reason while there is no need to separately benchmark bootstrap
>>>> commands.
>>>>
>>>> To summarize the above, try the following commands on LARGE datasets
>>>> (occupy e.g. half of your memory with data):
>>>> mlogit - you should see performance increase about 3 times on a 12
>>>> CPUs MP vs 1CPU IC.
>>>> summarize - you should see about 11-fold performance increase on a
>>>> 12CPUs MP vs 1CPU IC
>>>>
>>>> Run tests on a local machine. Perhaps it's the Amazon that is to blame
>>>> (I don't mean it). Some hosters limit your TOTAL computing power, so
>>>> you can get 128 cores with the same total performance as 1 core. Then
>>>> you are better of with a single CPU license of course :)
>>>>
>>>> Hope this helps.
>>>> Best, Sergiy Radyakin
>>>>
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>> *   http://www.ats.ucla.edu/stat/stata/
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/