st: Op. sys. refuses to provide memory - a cautionary tale

["Craig, Benjamin M." <Benjamin.Craig@moffitt.org>]

This posting is based on my recent experience with STATA MP 11 and may
be useful to those who wish to have more memory.

My modest six-month-old Dell PC has 4 GB RAM. Under Stata 9.2, I was
able to set memory to 935MB (see below), but when I installed Stata 11,
the memory was reduced by 50% (470MB). Adjusting the boot.ini or closing
programs had no effect, so I contacted the Stata help desk via email and
received the prompt and effective response (below).  

The end of the story is that after a day of tinkering I was able to
increase Stata MP 11 memory to 725MB by installing Windows Service Pack
3. And, I intend to buy a 64-bit machine, once budget allows.

I attribute this issue to Windows, not Stata; however, I hope that this
posting clarifies the issue for other Stata users, particularly those
purchasing a new machine or considering an upgrade to Stata 11.

Cheers,
Ben 

Under State 9.2
. qui for X in numlist 100(5)1000: capture set mem Xm . query mem

Current memory allocation

                    current                                 memory usage
    settable          value     description                 (1M = 1024k)
    --------------------------------------------------------------------
    set maxvar         5000     max. variables allowed           1.733M
    set memory          935M    max. data space                935.000M
    set matsize         400     max. RHS vars in models          1.254M
                                                            -----------
                                                               937.987M

Under Stata MP 11
. qui for X in numlist 100(5)1000: capture set mem Xm . query mem

Current memory allocation

                    current                                 memory usage
    settable          value     description                 (1M = 1024k)
    --------------------------------------------------------------------
    set maxvar         5000     max. variables allowed           1.909M
    set memory          470M    max. data space                470.000M
    set matsize         400     max. RHS vars in models          1.254M
                                                            -----------
                                                               473.163M

Under Stata MP 11 after installing Windows service pack 3
. qui for X in numlist 100(5)1000: capture set mem Xm . query mem

Current memory allocation

                    current                                 memory usage
    settable          value     description                 (1M = 1024k)
    --------------------------------------------------------------------
    set maxvar         5000     max. variables allowed           1.909M
    set memory          725M    max. data space                725.000M
    set matsize        6900     max. RHS vars in models        363.815M
                                                            -----------
                                                             1,090.724M

-----Original Message-----
From: Stata Technical Support [mailto:tech-support@stata.com] 
Sent: Monday, March 08, 2010 3:00 PM
To: Craig, Benjamin M.
Subject: Re: Op. sys. Refuses to provide memory 

Dear Ben,

What is your Windows XP service pack?  The problem you are describing
sounds like the memory issues that Windows XP Service pack 2 had until
Microsoft fixed it with the release of Service pack 3.  It is important
to remember that all memory management is handled by the operating
system.  Thus, when you try to increase the amount of RAM reserved for
Stata, it is the operating system that tells you it can't give you any
more memory.

Now besides the Windows XP Service pack 2 memory management issues, all
32-bit operating systems have trouble allocating large blocks of RAM to
any single application.  And since Stata requires a contiguous block of
RAM, meaning that the block of RAM must have contiguous adressing, this
issue becomes even more problematical.  A 32-bit operating system can
only theoretically allocate 2 GBs of RAM addressing to any one
application.  However, in practicality, the actual amount of continuous
RAM addressing that can be allocated to any one given application is
around 1 GB of RAM for a 32-bit Windows XP operating system.

**It is important to know that this upper allocation limit does not
increase by simply adding more RAM to the system because we are talking
about a contiguous block of RAM, and not just total available RAM.

As for Stata 9 having a larger block of RAM allocation that Stata 11
does, this is due to Stata 9's smaller footprint.  Since Stata 11 uses
more RAM just to be up and running, this translates into less of the
contiguous block of RAM that is leftover for loading datasets.

There are two options for increasing the amount of contiguous RAM that
can be allocated to Stata:

	1)  Go into the Windows Task manager and shutdown as many
unnecessary programs as you can in an attempt to free up more system
resources.

	2)  Load Stata 11 onto a 64-bit operating system, which won't
have the same memory restrictions that a 32-bit operating system has.



This transmission may be confidential or protected from disclosure and is only for review and use by the intended recipient. Access by anyone else is unauthorized. Any unauthorized reader is hereby notified that any review, use, dissemination, disclosure or copying of this information, or any act or omission taken in reliance on it, is prohibited and may be unlawful. If you received this transmission in error, please notify the sender immediately. Thank you.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/