RE: st: Stata SE 11.1 memory problem

["Liao, Junlin" <junlin-liao@uiowa.edu>]

I did some test on my system (Win7 64 bit ultimate with 4 GB physical memory). I can assign to Stata (11 SE) 2 or 3 GB or even beyond 4 GB. I set it to 4.5 GB and saw no ill effect on my computer. My guess is that Stata does not actuarially request the amount of memory you set. It only request the memory when it needs so. But the setting is a maximum just like everything else.

Junlin
________________________________________
From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] on behalf of Rebecca Pope [rebecca.a.pope@gmail.com]
Sent: Saturday, February 26, 2011 12:00 PM
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: Stata SE 11.1 memory problem

Just to recap:
1. You have a 64-bit OS (you still haven't told us which you are
using) _and_ 64-bit Stata - This was in one of the posts that Eric
sent, so I assume you checked it & this wasn't the problem.
2. Your 64-bit OS does not have any memory caps, again from Eric.
3. Your computer is recognizing all of the memory that is installed &
a sufficiently large amount of it is available for use.

I'm still trying to understand the sequence of events, because it
almost sounds like you've got two problems. You say in your original
post you can "only use memory upto 1300m in Stata." So, problem (1)
When you tell Stata -set mem 1500m- (here I'm just picking some number
greater than 1300), you are getting a message that says "op. sys.
refuses to provide memory". You don't report the error message, but
that's what I infer from the statement about only being able to use
1300m.

Problem (2), as I understand it, is that you've set your memory to the
maximum you can (1300 MB) but this is somehow insufficient for Stata
to read in a dataset that is 244 MB and merge it with a smaller
dataset (your reference to "no room to add observations"). This
frankly seems implausible to me and would be an interesting puzzle. Is
this what is happening?

Regards,
Rebecca



         __o                __o
      _`\ <,_            _`\ <,_
     (_)/   (_)          (_)/   (_)
=========================



2011/2/26  <figengunes@bilkent.edu.tr>:
> Thank you Eric and Rebecca for your replies. Stata gives me the error
> saying that there is no room to add observations. Big dataset is long. I
> have checked all you have said. I have enough memory but somehow stata can
> not use it. I will ask a tech support. Anyways I have managed to merge the
> datasets by dividing the big one into four pieces and then cleaning each
> merged piece before merging the next. It is not an ideal solution but
> works for now.
>
>
>  > To add to Eric's comments about operating systems:
>> I am using Windows 7 6-bit Professional, and when I first got my
>> computer, I couldn't access all of the memory. So, it is still
>> possible to not have access to all of the physical memory for your
>> computer, even when the operating system is _supposed_ to allow you to
>> access it. In my case, it turned out that something had gotten messed
>> up in the configuration process at the manufacturer and although I had
>> 8GB of memory installed, the Windows OS only recognized 2GB. Some
>> setting or something wasn't right; I don't pretend to understand how
>> Windows works/does not work with respect to how it recognizes and
>> allocates memory. However, tech support on my campus was able to do
>> something to fix it, so you might want to check with whomever provides
>> tech support for your computer if you find a problem.
>>
>> You don't say which OS you're using, but if it's Windows 7, here's how
>> you can check the amount of memory the computer recognizes: Control
>> panel > System and security > System
>>
>> It will tell you how much memory is installed and how much is
>> available for program use (i.e. not reserved for the OS). You might
>> want to make sure that all the memory you bought is recognized. Also,
>> check out the "Memory" tab of the Resource Manager (reached through
>> Task Manager) too. It will show you how much memory is in use, how
>> much is on "standby" (whatever that means), and how much is "free". In
>> my experience, Stata can take up to standby+free. There is a portion I
>> can never access and a portion that is marked as "modified". Even if
>> I've closed other programs, some of the "modified" remains unavailable
>> until I restart. Such is life with Windows.
>>
>> As an aside:
>> You say your large data set is 244MB and the other is "small",
>> presumably less than 244 MB. If that's the case, I don't see why you
>> should need to allocate more than 1300 MB. Stata doesn't necessarily
>> perform better with more memory
>> (http://www.stata.com/support/faqs/win/memory1.html).
>>
>> Regards,
>> Rebecca
>>
>>
>> On Fri, Feb 25, 2011 at 8:58 PM, Eric Booth <ebooth@ppri.tamu.edu> wrote:
>>> <>
>>> On Feb 25, 2011, at 6:50 PM, <figengunes@bilkent.edu.tr> wrote:
>>>
>>>> Thank you for your answer. I have 6GB RAM. I am running nothing else
>>>> than
>>>> Stata. I have the threads but no help.
>>>
>>> What type of OS are you running?  One of the threads I mentioned
>>> includes some mention about 64-bit OS's that still limit users to 2 or 4
>>> GB (e.g., Win7 Starter) (see:
>>>  http://www.stata.com/support/faqs/win/64bit.html   and
>>> http://msdn.microsoft.com/en-us/library/aa366778#physical_memory_limits_windows_7
>>>  ) .
>>> Also, you may have not opened any programs, but there are always things
>>> running in the background (again, _possibly_ enough to take up 700mb of
>>> memory and leave you ~ 1300mb to work with (?) ).   In the first thread
>>> link I provided, the Stata technical support response mentions how to
>>> look at your task manager (or your OS's equivalent)  to examine your
>>> system resources before starting Stata and while trying to set the
>>> memory in Stata.
>>>
>>>> One of the files I have is 244 Mb,
>>>> the other one is small. Is there a way to tell to the operating system
>>>> to
>>>> devote more memory to Stata? I am using very typical financial
>>>> databases,
>>>> therefore there is probably a way to work with these datasets.
>>>>
>>>
>>>  A few years ago, when I was running WinXP (with Stata 9 or 10), I had
>>> some luck getting  higher memory allocation by "stepping up" my memory
>>> -- try running this code and see if it hits the same 1300m ceiling:
>>>
>>> ********************!
>>> clear
>>> forval n = 100(100)6000 {
>>>        qui set mem `n'm
>>>        qui query mem
>>>        di in y  "`r(memory)'"
>>> }
>>> memory
>>> ********************!
>>> If you only need a subset of this data you could specify that in your
>>> -use- statement (e.g.,  - use var1-var5 using "myfile.dta", clear - ),
>>> but that doesn't solve your issue.
>>>
>>> __
>>> Eric A. Booth
>>> Public Policy Research Institute
>>> Texas A&M University
>>> ebooth@ppri.tamu.edu
>>>
>>>
>>>
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/statalist/faq
>>> *   http://www.ats.ucla.edu/stat/stata/
>>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged.  If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited.  Please reply to the sender that you have received the message in error, then delete it.  Thank you.
________________________________

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/