Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: file size problems


From   Jeph Herrin <[email protected]>
To   [email protected]
Subject   Re: st: file size problems
Date   Sun, 20 Aug 2006 12:39:13 -0400

This is very helpful - guess I've been -use-ing data for so many years,
it didn't occur to me to check -help use-.  My oversight.

Thanks,
Jeph




Scott Merryman wrote:
Regarding the third question: rather than subset the data after it is in
memory try loading only the relevant subset,
-use first.dta if month<mdy(1,1,2000)-

For example:

. set memory 1m

Current memory allocation

current memory usage
settable value description (1M = 1024k)
--------------------------------------------------------------------
set maxvar 5000 max. variables allowed 1.733M
set memory 1M max. data space 1.000M
set matsize 400 max. RHS vars in models 1.254M
-----------
3.987M

. sysuse auto
(1978 Automobile Data)

. expand 275
(20276 observations created)

. keep if fore ==1
no room to add more variables due to width
An attempt was made to add a variable that would have increased the
memory required to store an observation beyond what is currently possible.
You have the following alternatives:

1. Store existing variables more efficiently; see help compress.

2. Drop some variables or observations; see help drop. (Think of
Stata's data area as the area of a rectangle; Stata can trade off width and
length.)

3. Increase the amount of memory allocated to the data area using the
set memory command; see help memory.
r(902);

. save "C:\Documents and Settings\Scott_2\Desktop\temp.dta", replace
file C:\Documents and Settings\Scott_2\Desktop\temp.dta saved

. clear

. use "C:\Documents and Settings\Scott_2\Desktop\temp.dta" if fore == 1
(1978 Automobile Data)


Scott



-----Original Message-----
From: [email protected] [mailto:owner-
[email protected]] On Behalf Of Jeph Herrin
Sent: Sunday, August 20, 2006 10:50 AM
To: [email protected]
Subject: Re: st: file size problems

Sorry, perhaps I should have been more clear. I have 3 distinct questions:

1) Why does a 500mb file require 800mb of Stata memory?
2) How to get around "op. sys. refuses to supply memory" error
    when using -set mem 1000m- in winxp (when xp should be able
    to access at least 2.5gb, and no other apps are running)?
3) How to drop observations once the file is in memory, when
    -keep if mycondition- gives a "no room to add more obs" error
    (and there is no way to increase memory)?

thanks,
Jeph



*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index