Maybe you could try loading only the variables you need in the analysis. -odbc- allows you to choose colums.
That would reduce the required memory size. Unless you need all 200 variables.
Cheers
Manos
________________________________________
From: [email protected] [[email protected]] On Behalf Of natalie chan [[email protected]]
Sent: 13 July 2008 14:41
To: [email protected]
Subject: Re: st: R: memory problem when using odbe to read in large dataset
I have also tried to read in the dataset separately using "in":
odbc load in 1/60000, table ("firms")
odbc load in 60001/120000, table ("firms")
..
The first command works, I successfully read in the first 60000
observations but the second and other commands did not work, that is,
the Stata only read in the row of the variable names but no
observation was read in.
Have anyone met similar problem before?
Many thanks
On Sun, Jul 13, 2008 at 2:19 PM, Carlo Lazzaro <[email protected]> wrote:
>
> Dear Natalie,
> I would suggest you to take a look at - help compress -.
>
> HTH and Kind Regards,
>
> Carlo
> -----Messaggio originale-----
> Da: [email protected]
> [mailto:[email protected]] Per conto di natalie chan
> Inviato: domenica 13 luglio 2008 14.27
> A: [email protected]
> Oggetto: st: memory problem when using odbe to read in large dataset
>
> Dear Statalisters,
>
> I am trying to read in a dataset with more than 200 variables and 2
> million observations. the size of the dataset is 2.1G. I am using
> Stata/MP 10 and Ram of the computer I am using is 1G
>
> I set the memory to 500m. the command I used was:
>
> odbc load, table ("firms")
>
> The stata produced the message "obs must be
> between 62593 and 62593. As a possible solution, try increasing memory
> using set memory ".
>
> I increased the memory to 700m, which is the maximum possible memeory
> I can use, but still it did not work.
>
> Could anyone here give some advice? Thanks a lot.
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
>
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
The University of Aberdeen is a charity registered in Scotland, No SC013683.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
