Re: st: R: memory problem when using odbe to read in large dataset

["natalie chan" <natalie.channew@googlemail.com>]

Dear Manos,

Thanks for your help. I will try that too.

Many thanks.

Have a good day.

Natalie

On Sun, Jul 13, 2008 at 3:23 PM, Mentzakis, Emmanouil
<e.mentzakis@abdn.ac.uk> wrote:
> Maybe you could try loading only the variables you need in the analysis. -odbc- allows you to choose colums.
>
> That would reduce the required memory size. Unless you need all 200 variables.
>
> Cheers
> Manos
>
> ________________________________________
> From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] On Behalf Of natalie chan [natalie.channew@googlemail.com]
> Sent: 13 July 2008 14:41
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: R: memory problem when using odbe to read in large dataset
>
> I have also tried to read in the dataset separately using "in":
>
> odbc load in 1/60000, table ("firms")
> odbc load in 60001/120000, table ("firms")
> ..
>
> The first command works, I successfully read in the first 60000
> observations but the second and other commands did not work, that is,
> the Stata only read in the row of the variable names but no
> observation was read in.
>
> Have anyone met similar problem before?
>
> Many thanks
>
> On Sun, Jul 13, 2008 at 2:19 PM, Carlo Lazzaro <carlo.lazzaro@tin.it> wrote:
>>
>> Dear Natalie,
>> I would suggest you to take a look at - help compress -.
>>
>> HTH and Kind Regards,
>>
>> Carlo
>> -----Messaggio originale-----
>> Da: owner-statalist@hsphsun2.harvard.edu
>> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di natalie chan
>> Inviato: domenica 13 luglio 2008 14.27
>> A: statalist@hsphsun2.harvard.edu
>> Oggetto: st: memory problem when using odbe to read in large dataset
>>
>> Dear Statalisters,
>>
>> I am trying to read in a dataset with more than 200 variables and 2
>> million observations. the size of the dataset is 2.1G. I am using
>> Stata/MP 10 and Ram of the computer I am using is 1G
>>
>> I set the memory to 500m. the command I used was:
>>
>> odbc load, table ("firms")
>>
>> The stata produced the message "obs must be
>> between 62593 and 62593. As a possible solution, try increasing memory
>> using  set memory ".
>>
>> I increased the memory to 700m, which is the maximum possible memeory
>> I can use, but still it did not work.
>>
>>  Could anyone here give some advice? Thanks a lot.
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/support/faqs/res/findit.html
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/support/faqs/res/findit.html
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
>
> The University of Aberdeen is a charity registered in Scotland, No SC013683.
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/