Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: Op. sys. refuses to provide memory - a cautionary tale
From
"Craig, Benjamin M." <[email protected]>
To
<[email protected]>
Subject
RE: st: Op. sys. refuses to provide memory - a cautionary tale
Date
Thu, 19 Aug 2010 15:48:21 -0400
To clarify, the website answers essential questions (which systems are supported?) and provides some advice and education. I need to know more details on current controversies relating to multiple core, 64-bit and drive speeds in the real world. In this sense, it is a bit incomplete to say that more core, more bits, and faster drives are preferable. As was previously post, I had thought that a 32-bit dual core desktop was good enough for my needs, and was woefully wrong. Others seem to follow...
http://www.stata.com/statalist/archive/2010-07/msg01337.html
If a listserv member has tried STATA MP on multiple machines, I would like to know what worked best so that I can buy one. There is an obvious caveat: speed depends on the task. However, I would counter that some data are better than none. For example, I recently bootstrapped a ML with 1000 iteration and inequality constraints. It took 4 weeks. Do you think your machine can do better? Personally, I do not use stata for database management, and doubt that many econometricians do. If someone has a good analytics machine, and he/she thinks that it works well, I'd like to know its components. Maybe a consensus will emerge. Maybe not.
Ben
Benjamin M. Craig, PhD
Assistant Member, Health Outcomes & Behavior, Moffitt Cancer Center
Associate Professor, Economics, University of South Florida
12902 Magnolia Dr, MRC-CANCONT, Tampa, FL 33612-9416
Phone (813) 745-6710; Fax (813) 745-6525
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Martin Weiss
Sent: Thursday, August 19, 2010 3:04 PM
To: [email protected]
Subject: RE: st: Op. sys. refuses to provide memory - a cautionary tale
<>
In which respect is the Stata website "incomplete"? There is advice at http://www.stata.com/products/opsysmp.html, and how is the website supposed to give more detailed advice? It does not know your specific setup, hence the reluctance to go into greater depth...
HTH
Martin
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Craig, Benjamin M.
Sent: Donnerstag, 19. August 2010 20:48
To: [email protected]
Subject: RE: st: Op. sys. refuses to provide memory - a cautionary tale
Okay, I give up... I need a new machine. Due to institutional policies, I need to buy a Dell. Otherwise, I would very much like any advice on this purchase. My best guess is a 64-bit 8-core desktop for a 6-core version of Stata MP. I don't need a rocket, just a racecar.
If you have any specifications that you would like to share with me, please send them directly or post them on the listserv for others looking to upgrade. I have read the Stata website, which seems incomplete.
Cheers,
Ben
Benjamin M. Craig, PhD
Assistant Member, Health Outcomes & Behavior, Moffitt Cancer Center
Associate Professor, Economics, University of South Florida
12902 Magnolia Dr, MRC-CANCONT, Tampa, FL 33612-9416
Phone (813) 745-6710; Fax (813) 745-6525
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Craig, Benjamin M.
Sent: Tuesday, March 09, 2010 2:30 PM
To: [email protected]
Subject: RE: st: Op. sys. refuses to provide memory - a cautionary tale
Greater follow-up with tech support rendered the following insightful response (below). In summary, a 64-bit machine is required to take full advantage of Stata MP 11. At least, I have a clear justification to loosen the purse strings of my IT department.
-----Original Message-----
From: Stata Technical Support [mailto:[email protected]]
Sent: Tuesday, March 09, 2010 1:55 PM
To: Craig, Benjamin M.
Subject: Re: RE: RE: Op. sys. Refuses to provide memory
Stata 11 has a much bigger operational footprint than Stata 9 does, and as such Stata 11 uses much more of your system's resources than Stata 9. This difference is especially true when using commands that use Factor Variables, such as your
. logistic rct upper if ((tri_cat==1)+(tri-cat==3))
command.
Thus, when you run the above command in Stata 9, the command runs in Stata, using Stata functions and commands (which was optimal behavior for 32-bit systems). But when you run the above command in Stata 11, the command is run in Mata, using Mata functions and commands (which is optimal behavior for modern 64-bit systems that don't have the memory limitations that legacy 32-bit systems do). This difference allows Stata 11 to be much faster than Stata 9, but since it has a much bigger operational footprint than Stata 9, it requires more system resources (cpu time, RAM resources, etc...).
Therefore, the r(909) error message you get after running the above command in Stata 11 is caused by Stata 11 needing more RAM resources so Stata can run the above command in Mata. But, since there aren't enough available resources that the operating system can allocate to Mata, so that Stata can run the command in Mata, you get the r(909) error.
You might try running your do-file using -version control- in Stata 11.
This way, you can tell Stata 11 to run like Stata 9, which might allow your system to preserve enough resources so it can successfully run the above command.
This may not work because Stata 11 uses so much of your system resources to begin with, but just in case there are enough resources still available, you can try using the -version control- command. For example, at the top of your do-file, include the following code:
**************************************************************************
. version 9.2
. clear
. set more off
. qui for X in numlist 100(25)1000: capture set mem Xm
. qui for X in numlist 500(25)10000: capture set matsize X
. query mem
...
**************************************************************************
Again, this may or may not work. It all depends on how much of your RAM is available for Stata's use as allocated by the operating system. Type -help
version- in the Stata Command window for more information and syntax usage.
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Tony
Sent: Tuesday, March 09, 2010 1:38 PM
To: [email protected]
Subject: Re: st: Op. sys. refuses to provide memory - a cautionary tale
I have 16GB memory with 64-bit Windows 7 and this is not a problem.
It must be the outdated setup.
Tony
On Tue, Mar 9, 2010 at 4:16 AM, Craig, Benjamin M.
<[email protected]> wrote:
> This posting is based on my recent experience with STATA MP 11 and may
> be useful to those who wish to have more memory.
>
> My modest six-month-old Dell PC has 4 GB RAM. Under Stata 9.2, I was
> able to set memory to 935MB (see below), but when I installed Stata
> 11, the memory was reduced by 50% (470MB). Adjusting the boot.ini or
> closing programs had no effect, so I contacted the Stata help desk via
> email and received the prompt and effective response (below).
>
> The end of the story is that after a day of tinkering I was able to
> increase Stata MP 11 memory to 725MB by installing Windows Service
> Pack 3. And, I intend to buy a 64-bit machine, once budget allows.
>
> I attribute this issue to Windows, not Stata; however, I hope that
> this posting clarifies the issue for other Stata users, particularly
> those purchasing a new machine or considering an upgrade to Stata 11.
>
> Cheers,
> Ben
>
> Under State 9.2
> . qui for X in numlist 100(5)1000: capture set mem Xm . query mem
>
> Current memory allocation
>
> current memory
> usage
> settable value description (1M =
> 1024k)
>
> --------------------------------------------------------------------
> set maxvar 5000 max. variables allowed 1.733M
> set memory 935M max. data space 935.000M
> set matsize 400 max. RHS vars in models 1.254M
> -----------
> 937.987M
>
> Under Stata MP 11
> . qui for X in numlist 100(5)1000: capture set mem Xm . query mem
>
> Current memory allocation
>
> current memory
> usage
> settable value description (1M =
> 1024k)
>
> --------------------------------------------------------------------
> set maxvar 5000 max. variables allowed 1.909M
> set memory 470M max. data space 470.000M
> set matsize 400 max. RHS vars in models 1.254M
> -----------
> 473.163M
>
> Under Stata MP 11 after installing Windows service pack 3 . qui for X
> in numlist 100(5)1000: capture set mem Xm . query mem
>
> Current memory allocation
>
> current memory
> usage
> settable value description (1M =
> 1024k)
>
> --------------------------------------------------------------------
> set maxvar 5000 max. variables allowed 1.909M
> set memory 725M max. data space 725.000M
> set matsize 6900 max. RHS vars in models 363.815M
> -----------
> 1,090.724M
>
> -----Original Message-----
> From: Stata Technical Support [mailto:[email protected]]
> Sent: Monday, March 08, 2010 3:00 PM
> To: Craig, Benjamin M.
> Subject: Re: Op. sys. Refuses to provide memory
>
> Dear Ben,
>
> What is your Windows XP service pack? The problem you are describing
> sounds like the memory issues that Windows XP Service pack 2 had until
> Microsoft fixed it with the release of Service pack 3. It is
> important to remember that all memory management is handled by the
> operating system. Thus, when you try to increase the amount of RAM
> reserved for Stata, it is the operating system that tells you it can't
> give you any more memory.
>
> Now besides the Windows XP Service pack 2 memory management issues,
> all 32-bit operating systems have trouble allocating large blocks of
> RAM to any single application. And since Stata requires a contiguous
> block of RAM, meaning that the block of RAM must have contiguous
> adressing, this issue becomes even more problematical. A 32-bit
> operating system can only theoretically allocate 2 GBs of RAM
> addressing to any one application. However, in practicality, the
> actual amount of continuous RAM addressing that can be allocated to
> any one given application is around 1 GB of RAM for a 32-bit Windows
> XP
operating system.
>
> **It is important to know that this upper allocation limit does not
> increase by simply adding more RAM to the system because we are
> talking about a contiguous block of RAM, and not just total available RAM.
>
> As for Stata 9 having a larger block of RAM allocation that Stata 11
> does, this is due to Stata 9's smaller footprint. Since Stata 11 uses
> more RAM just to be up and running, this translates into less of the
> contiguous block of RAM that is leftover for loading datasets.
>
> There are two options for increasing the amount of contiguous RAM that
> can be allocated to Stata:
>
> 1) Go into the Windows Task manager and shutdown as many
> unnecessary programs as you can in an attempt to free up more system
> resources.
>
> 2) Load Stata 11 onto a 64-bit operating system, which won't
> have the same memory restrictions that a 32-bit operating system has.
>
>
>
> This transmission may be confidential or protected from disclosure and
> is
only for review and use by the intended recipient. Access by anyone else is unauthorized. Any unauthorized reader is hereby notified that any review, use, dissemination, disclosure or copying of this information, or any act or omission taken in reliance on it, is prohibited and may be unlawful. If you received this transmission in error, please notify the sender immediately.
Thank you.
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
This transmission may be confidential or protected from disclosure and is only for review and use by the intended recipient. Access by anyone else is unauthorized. Any unauthorized reader is hereby notified that any review, use, dissemination, disclosure or copying of this information, or any act or omission taken in reliance on it, is prohibited and may be unlawful. If you received this transmission in error, please notify the sender immediately.
Thank you.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
This transmission may be confidential or protected from disclosure and is only for review and use by the intended recipient. Access by anyone else is unauthorized. Any unauthorized reader is hereby notified that any review, use, dissemination, disclosure or copying of this information, or any act or omission taken in reliance on it, is prohibited and may be unlawful. If you received this transmission in error, please notify the sender immediately.
Thank you.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
This transmission may be confidential or protected from disclosure and is only for review and use by the intended recipient. Access by anyone else is unauthorized. Any unauthorized reader is hereby notified that any review, use, dissemination, disclosure or copying of this information, or any act or omission taken in reliance on it, is prohibited and may be unlawful. If you received this transmission in error, please notify the sender immediately. Thank you.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/