RE: st: Op. sys. refuses to provide memory - a cautionary tale

[Kiss Sándor Csanád <kiss.sandor.csanad@mkkt.hu>]

Sorry guys, I read all this, and also this

http://www.stata.com/statalist/archive/2010-01/msg00409.html

but what is the final word? How much memory can a 64bit system handle?

Thanks



          Kiss Sándor Csanád
          Elemző közgazdász / Economist
          Office of the Fiscal Council, Hungary
          1055 Budapest, Honvéd utca 20.
          T: (+36 1) 510 3025
          Mobil: (+36 30) 703 1024
          Fax: (+36 1) 510 3099
          Web: www.mkkt.hu


-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Jeph Herrin
Sent: Monday, August 23, 2010 7:30 PM
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: Op. sys. refuses to provide memory - a cautionary tale

I took the plunge on Windows 7 64bit and have been very
satisfied with it relative to XP 64bit, which is what that machine
was running. Though with only 16gb RAM, I haven't really put
it to the test you envision with 192gb.

For what it's worth, I shop my PCs from a custom builder that
caters to engineers and CAD/CAM types; they were dead set against
selling Vista but said that Win7 had so far proved itself for
their customers and they were recommending it for those who
needed not-Linux.

cheers,
Jeph


On 8/23/2010 12:45 PM, Jay Tuthill wrote:
> Hi All,
>
> I'm also in the middle of upgrading my hardware as my current 32 bit
> computers will not let me run very efficiently the datasets I want. I'm
> considering the new Intel workstation system SC5650SCWSR which offers
> dual Xeon processors and up to 192GB of RAM. Am curious if anyone has
> compared the Windows 7 64bit versus Window Server 2008 R2 64bit
> operating systems. (I have built my own computers for several years and
> have a wide latitude in how I configure them.)
>
> Thanks...Jay
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Eric Booth
> Sent: Saturday, August 21, 2010 5:24 PM
> To:<statalist@hsphsun2.harvard.edu>
> Subject: Re: st: Op. sys. refuses to provide memory - a cautionary tale
>
> <>
>
> On Aug 21, 2010, at 4:19 PM, Eric Booth wrote:
>
>>   but beyond this limit Stata won't slow down as you add or allocate
> more RAM.
>
> Clarification:
> That should say, "but UP TO (or before you reach this memory limit)
> Stata won't slow down as you add or allocate more RAM..."
>
> On Aug 21, 2010, at 4:19 PM, Eric Booth wrote:
>
>> <>
>> On Aug 20, 2010, at 4:07 PM, Tony wrote:
>>> Too much RAM will slow it down.
>>
>> Stata will certainly slow down if you set and use more memory in Stata
> than is physically available on your machine because you start using
> virtual memory, but beyond this limit Stata won't slow down as you add
> or allocate more RAM.
>> That is, it will take the same time to run a do-file on a 1G dataset
> whether you allocate 2G, 8G, or 20G of memory to Stata; however it will
> get excruciatingly slow if you set your memory to 500m and  use a 1G
> dataset.
>>
>>
>>> On Fri, Aug 20, 2010 at 1:28 PM, Craig, Benjamin M. wrote:
>>
>>>> The purpose is real world speed, so has anyone actually noticed if
> going
>>>> up to 24GB RAM, solid state drive expedited your jobs
>>
>>
>> I haven't tested the idea of getting a SSD drive, but I think the
> speed advantage would be evident mainly when you were opening(reading)
> or saving(writing) a large dataset since your using the data in memory
> the rest of the time.  I do want to try out installing a SSD drive for
> working with data that is larger than my physical RAM and requires me to
> use virtual memory to work with it (I've maxed out my physical RAM with
> 8G sticks in each of the slots).  I've read about moving your page/swap
> file to a SSD which should speed things up when working in virtual
> memory (but since SSDs wear out faster with more read/writes, this might
> be a concern).  Also, you could move the location of the tempfiles that
> Stata creates to that path by setting your OS system temp file location
> (see:  http://www.stata.com/statalist/archive/2009-05/msg00416.html).
> Maybe someone here has tried working with SSD and large datasets ?
>>
>> Again, more RAM is always better IMO--but it only speeds you up in the
> sense that it prevents you from using page file.  There are also speeds
> associated with RAM (mine is 1066 DDR3), but I don't know much
> differences in memory speed matters.
>>
>>
>>> On Fri, Aug 20, 2010 at 1:28 PM, Craig, Benjamin M. wrote:
>>>> Does any listserv member think that I should go from 6 to 12 cores?
>>>>
>>>> Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
>>>> Dual Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
>>
>> Depends on what you are doing.  If you've got a time intensive
> procedure that you're running on your 6 core machine, try running it
> with your -set processors- at 1, 2, 4, and 6 and see what kind of speed
> increase you observe, e.g.:
>>
>> *****!
>> timer clear
>> forval n = 1(2)6 {
>> clear all
>> set mem 32g
>> set processors `n'
>> timer on `n'
>> 	<your command goes here>
>> timer off `n'
>> 	}
>> timer list
>> *****!
>>
>> ~ Eric
>>
>>
>> __
>> Eric A. Booth
>> Public Policy Research Institute
>> Texas A&M University
>> ebooth@ppri.tamu.edu
>> Office: +979.845.6754
>>
>>
>>>
>>> On Fri, Aug 20, 2010 at 1:28 PM, Craig, Benjamin M.
>>> <Benjamin.Craig@moffitt.org>  wrote:
>>>> Thanks Nick, I have learned that to truly take advantage of the
> latest
>>>> version of Stata, 64-bits and 4 or more cores is required. To be a
> bit
>>>> more specific, lets assume I am using 6-core Stata MP on Windows 7
>>>> Professional, 64-bit for computationally intensive simulation
> analyses.
>>>>
>>>> Does any listserv member think that I should go from 6 to 12 cores?
>>>>
>>>> Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
>>>> Dual Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
>>>>
>>>> Is it worthwhile to upgrade from RAM and Hard drive? For example,
>>>>
>>>> 12GB DDR3 ECC SDRAM Memory,1333MHz,6X2GB
>>>>
>>>> 1TB SATA 3.0Gb/s,7200 RPM HardDrive with 32MB DataBurst Cache
>>>>
>>>> The purpose is real world speed, so has anyone actually noticed if
> going
>>>> up to 24GB RAM, solid state drive expedited your jobs? In theory, it
>>>> should, but I am hoping that someone has purchase a computer
> recently to
>>>> test this hypothesis.
>>>>
>>>> Cheers,
>>>> Ben
>>>>
>>>>
>>>> Benjamin M. Craig, PhD
>>>>
>>>> Assistant Member, Health Outcomes&  Behavior, Moffitt Cancer Center
>>>>
>>>> Associate Professor, Economics, University of South Florida
>>>>
>>>> 12902 Magnolia Dr, MRC-CANCONT, Tampa, FL 33612-9416
>>>>
>>>> Phone (813) 745-6710; Fax (813) 745-6525
>>>>
>>>> -----Original Message-----
>>>> From: owner-statalist@hsphsun2.harvard.edu
>>>> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Nick Cox
>>>> Sent: Friday, August 20, 2010 5:00 AM
>>>> To: 'statalist@hsphsun2.harvard.edu'
>>>> Subject: RE: st: Op. sys. refuses to provide memory - a cautionary
> tale
>>>>
>>>> I think this partially answers itself in that I don't think that it
> can
>>>> fairly be expected that a company website is a proper place for a
>>>> company, in this case StataCorp, to offer opinions about anything
> that
>>>> is currently controversial.
>>>>
>>>> That said, Benjamin's question is obviously practical and a fair one
> for
>>>> members of this list to venture opinions and comment from
> experience. In
>>>> addition, presumably people other than econometricians are not
> excluded.
>>>>
>>>>
>>>> Nick
>>>> n.j.cox@durham.ac.uk
>>>>
>>>> Craig, Benjamin M.
>>>>
>>>> To clarify, the website answers essential questions (which systems
> are
>>>> supported?) and provides some advice and education. I need to know
> more
>>>> details on current controversies relating to multiple core, 64-bit
> and
>>>> drive speeds in the real world. In this sense, it is a bit
> incomplete to
>>>> say that more core, more bits, and faster drives are preferable. As
> was
>>>> previously post, I had thought that a 32-bit dual core desktop was
> good
>>>> enough for my needs, and was woefully wrong. Others seem to
> follow...
>>>>
>>>> http://www.stata.com/statalist/archive/2010-07/msg01337.html
>>>>
>>>> If a listserv member has tried STATA MP on multiple machines, I
> would
>>>> like to know what worked best so that I can buy one. There is an
> obvious
>>>> caveat: speed depends on the task. However, I would counter that
> some
>>>> data are better than none. For example, I recently bootstrapped a ML
>>>> with 1000 iteration and inequality constraints. It took 4 weeks. Do
> you
>>>> think your machine can do better? Personally, I do not use stata for
>>>> database management, and doubt that many econometricians do. If
> someone
>>>> has a good analytics machine, and he/she thinks that it works well,
> I'd
>>>> like to know its components. Maybe a consensus will emerge. Maybe
> not.
>>>>
>>>> Martin Weiss
>>>>
>>>> In which respect is the Stata website "incomplete"? There is advice
> at
>>>> http://www.stata.com/products/opsysmp.html, and how is the website
>>>> supposed to give more detailed advice? It does not know your
> specific
>>>> setup, hence the reluctance to go into greater depth...
>>>>
>>>> Craig, Benjamin M.
>>>>
>>>> Okay, I give up... I need a new machine. Due to institutional
> policies,
>>>> I need to buy a Dell. Otherwise, I would very much like any advice
> on
>>>> this purchase. My best guess is a 64-bit 8-core desktop for a 6-core
>>>> version of Stata MP. I don't need a rocket, just a racecar.
>>>>
>>>> If you have any specifications that you would like to share with me,
>>>> please send them directly or post them on the listserv for others
>>>> looking to upgrade. I have read the Stata website, which seems
>>>> incomplete.
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/