Re: st: file size problems

[Jeph Herrin <junk@spandrel.net>]

Thanks for the link, which answers my question #2. Here's hoping for a
robust 64-bit XP some day (how I miss Linux...).


Nuno Soares wrote:
> Try reading the link http://www.stata.com/support/faqs/win/winmemory.html
> for eventual reasons for this behaviour... However, I have a system with
> 1.5gb, winxp, and Stata mem defined to 900m and it loads a file with 680mb
> with no problems at all (it has 3400000 observations and 57 vars). 
> Best,
>
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Jeph Herrin
> Sent: 20 August 2006 16:50
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: file size problems
>
> Sorry, perhaps I should have been more clear. I have 3 distinct questions:
>
> 1) Why does a 500mb file require 800mb of Stata memory?
> 2) How to get around "op. sys. refuses to supply memory" error
>     when using -set mem 1000m- in winxp (when xp should be able
>     to access at least 2.5gb, and no other apps are running)?
> 3) How to drop observations once the file is in memory, when
>     -keep if mycondition- gives a "no room to add more obs" error
>     (and there is no way to increase memory)?
>
> thanks,
> Jeph
>
>
> b. water wrote:
>
> > how about using the set memory to increase Stata memory allocation? have
> you tried this?
> > bw
> >
> >
> >
> >
> > ----------------------------------------
> > > Date: Sat, 19 Aug 2006 19:36:48 -0400
> > > From: junk@spandrel.net
> > > To: statalist@hsphsun2.harvard.edu
> > > Subject: st: file size problems
> > >
> > > I'm using Stata 9.2 on WinXP (SP2), updated.
> > >
> > > I have a file which is 500mb, but when I load it into Stata, it 
> > > requires 825mb. That doesn't seem right:
> > >
> > > . dir first.dta
> > >   500.7M   8/19/06 18:33  first.dta
> > >
> > > . u first.dta, clear
> > >
> > > . d
> > >
> > > Contains data from first.dta
> > >    obs:    75,006,197
> > >   vars:             3                          19 Aug 2006 18:33
> > >   size:   825,068,167 (12.6% of memory free)
> > > ---------------------------------------------------
> > >
> > >
> > > This is a problem, because WinXp will only allocate 900mb to Stata 
> > > (anyone know how to change this? the machine has 4gb). The first 
> > > thing I want to do when I load this file is:
> > >
> > > . keep if month<mdy(1,1,2000)
> > >
> > > however:
> > >
> > > no room to add more variables due to width
> > >      An attempt was made to add a variable that would have increased 
> > > the memory required to store an observation beyond
> > >      what is currently possible.  You have the following alternatives:
> > >
> > >       1.  Store existing variables more efficiently; see help compress.
> > >
> > >       2.  Drop some variables or observations; see help drop.  (Think 
> > > of Stata's data area as the area of a rectangle;
> > >           Stata can trade off width and length.)
> > >
> > >       3.  Increase the amount of memory allocated to the data area 
> > > using the set memory command; see help memory.
> > > r(902);
> > >
> > >
> > >
> > > I can't drop any other variables (note that there are only 3). Any 
> > > ideas? In particular, is there an alternative to "keep" that doesn't add
> a variable?
> > > thanks,
> > > Jeph
> > > *
> > > *   For searches and help try:
> > > *   http://www.stata.com/support/faqs/res/findit.html
> > > *   http://www.stata.com/support/statalist/faq
> > > *   http://www.ats.ucla.edu/stat/stata/
> > _________________________________________________________________
> > Try Live.com - your fast, personalized homepage with all the things you
> care about in one place.
> > http://www.live.com/getstarted
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/support/faqs/res/findit.html
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/