Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: Large data sets


From   "Svend Juul" <[email protected]>
To   <[email protected]>
Subject   Re: st: Large data sets
Date   Sat, 23 Jun 2007 10:58:20 +0200

Carmen wrote:
 
I need to work with large datasets (>700M, <2g), but Stata 9.2 does not
allow me to set memory above 500M (my computer should allow for up to 2g
memory setting).  
...
 
------------------------------------------------------
 
Michael Blasnik gave some advice, including advice on how to reduce the
size of the dataset. This, however, requires the dataset to be opened
by Stata, and if the magnitude of the dataset forbids that, you have a
problem. 
 
One possibility is to spit the dataset in parts, then make the reductions,
and finally combine the reduced datasets. I don't know if this could help
Carmen: 
 
      use "flyt.dta" in 1/1000000
      compress
      keep var1-var17
      save flyt1.dta , replace
 
      use "flyt.dta" in 1000001/2000000
      compress
      keep var1-var17
      save flyt2.dta , replace
 
      use "flyt.dta" in 2000001/l    // l is lowercase L
      compress
      keep var1-var17
      save flyt3.dta , replace
 
      use flyt1.dta, clear
      append using flyt2.dta
      append using flyt3.dta
 

Svend
________________________________________________________ 
 
Svend Juul
Institut for Folkesundhed, Afdeling for Epidemiologi
(Institute of Public Health, Department of Epidemiology)
Vennelyst Boulevard 6 
DK-8000 Aarhus C,  Denmark 
Phone, work:  +45 8942 6090 
Phone, home:  +45 8693 7796 
Fax:          +45 8613 1580 
E-mail:       [email protected] 
_________________________________________________________ 

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index