Re: st: file size problems

[Jeph Herrin <junk@spandrel.net>]

Sorry, perhaps I should have been more clear. I have 3 distinct questions:

1) Why does a 500mb file require 800mb of Stata memory?
2) How to get around "op. sys. refuses to supply memory" error
   when using -set mem 1000m- in winxp (when xp should be able
   to access at least 2.5gb, and no other apps are running)?
3) How to drop observations once the file is in memory, when
   -keep if mycondition- gives a "no room to add more obs" error
   (and there is no way to increase memory)?

thanks,
Jeph


b. water wrote:
> how about using the set memory to increase Stata memory allocation? have you tried this?
>
> bw
>
>
>
>
> ----------------------------------------
> > Date: Sat, 19 Aug 2006 19:36:48 -0400
> > From: junk@spandrel.net
> > To: statalist@hsphsun2.harvard.edu
> > Subject: st: file size problems
> >
> > I'm using Stata 9.2 on WinXP (SP2), updated.
> >
> > I have a file which is 500mb, but when I load it into
> > Stata, it requires 825mb. That doesn't seem right:
> >
> > . dir first.dta
> >   500.7M   8/19/06 18:33  first.dta
> >
> > . u first.dta, clear
> >
> > . d
> >
> > Contains data from first.dta
> >    obs:    75,006,197
> >   vars:             3                          19 Aug 2006 18:33
> >   size:   825,068,167 (12.6% of memory free)
> > ---------------------------------------------------
> >
> >
> > This is a problem, because WinXp will only allocate 900mb to Stata (anyone
> > know how to change this? the machine has 4gb). The first thing I want to do
> > when I load this file is:
> >
> > . keep if month<mdy(1,1,2000)
> >
> > however:
> >
> > no room to add more variables due to width
> >      An attempt was made to add a variable that would have increased the memory 
> > required to store an observation beyond
> >      what is currently possible.  You have the following alternatives:
> >
> >       1.  Store existing variables more efficiently; see help compress.
> >
> >       2.  Drop some variables or observations; see help drop.  (Think of Stata's 
> > data area as the area of a rectangle;
> >           Stata can trade off width and length.)
> >
> >       3.  Increase the amount of memory allocated to the data area using the set 
> > memory command; see help memory.
> > r(902);
> >
> >
> >
> > I can't drop any other variables (note that there are only 3). Any ideas? In
> > particular, is there an alternative to "keep" that doesn't add a variable?
> >
> > thanks,
> > Jeph
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/support/faqs/res/findit.html
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> _________________________________________________________________
> Try Live.com - your fast, personalized homepage with all the things you care about in one place.
> http://www.live.com/getstarted
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/