Re: st: RE: Efficient parallel computing in Stata/MP

[Demian Panigo <panigo@gmail.com>]

Thanks a lot. I need to run many linear and panel regressions (with
many associated tests) but over a rather medium to small database. So,
you are wright, multiple Stata instances (even Stata SE or IC
instances) will help more than splitting the dataset into multiple
clusters (as Stata MP does).
Notwithstanding, some additional time reduction can be obtained by
means of Stata/MP but I only have an 8-core CPU available for private
use. Just like you, I'm not sure about how many cores I need to take
full advantage of each additional Stata/MP instance.
I hope some guy in Stata corporation could help me with that.
Thanks again
Demian



2013/9/26 Timothy Mak <tshmak@hku.hk>:
> It should be borne in mind that the speed increase in using MP (as opposed to SE or IC) is very often only significant if you're analysing a large dataset, especially for something as simple as -regress-. Opening multiple instances of Stata, however, does not have that limitation, as long as your computer has multiple cores. Also, I'm unsure whether you need 16 cores to fully utilize the MP power if you were to run 4 instances of Stata with MP4 capabilities. Maybe Statacorp would be able to comment on that more. See also -parallel- from SSC for a possible shortcut in parallelizing your work.
>
> Tim
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Demian Panigo
> Sent: 27 September 2013 01:02
> To: statalist@hsphsun2.harvard.edu
> Subject: st: Efficient parallel computing in Stata/MP
>
> Dear Statalist members: I need some help, because I'm not sure about
> some Stata/MP properties for parallel computing.
> We know from http://www.stata.com/statamp/statamp.pdf that many
> estimation commands (e.g. regress) are almost fully parallelizable and
> that average efficiency for all commands is around 72%. So, in
> standard linear regression problems (e.g running one million equations
> for parameter stability analysis), using Stata/MP in a multiple-core
> CPU would be an optimal time saving strategy.
> However, it is also possible to exploit the multi-core CPU environment
> by working with multiple parallel Stata/MP instances (e.g. using 4
> parallel Stata/MP instances to run 250.000 linear regressions with
> each core).
> My question is simple.... Can I save some time by using this "dual
> parallelism" methodology? (because parallel computing is
> authomatically used by Stata/MP to parallelize internal tasks of, for
> example, regress; and because I also parallelize the whole set of
> regressions between 4 cores, by means of multiple Stata/MP instances).
> Thanks in advance
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/



-- 
Demian T. Panigo
Lic. en Economía, UNLP,
Master en Cs Sociales, UBA,
Doctor en Economía, EHESS-ENS (Paris)
Investigador Adjunto del CEIL-PIETTE del CONICET
Docente investigador de la UNM, de la UNLP, de la UBA, y de
Paris-Jourdan Sciences Economiques-ENS.
Miembro del Programa de Formación Popular en Economía (PROFOPE)

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/