Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: file too big to import
From
Eric Booth <[email protected]>
To
"<[email protected]>" <[email protected]>
Subject
Re: st: file too big to import
Date
Wed, 20 Apr 2011 12:00:07 +0000
<>
I guess that's assuming you don't have Small Stata (which is limited to 1200 observations -- see -help limits-). If you're still having issues getting this imported, more information about your system (version of Stata, OS, physical RAM) would be helpful for others to answer your question.
- Eric
__
Eric A. Booth
Public Policy Research Institute
Texas A&M University
[email protected]
Office: +979.845.6754
On Apr 20, 2011, at 6:54 AM, Eric Booth wrote:
> <>
>
> First, try increasing your memory -- if you're setting your mem with "set mem 10000", that's setting it to 10000K, not 10000M (examine -query mem- or -memory- for details). Even so, try "stepping up" your memory as much as possible. Here is a good guide to the amount of mem you'll need: http://www.stata.com/support/faqs/data/howbig.html Most modern machines (even 32-bit) should be able to handle data the size you are describing.
>
> Second, you can try using -infile- (see this thread on some tips: http://www.stata.com/statalist/archive/2003-09/msg00122.html). Also, the user program -chunky- (from SSC) may help. Finally, you could always try the software StatTransfer (http://www.stattransfer.com/).
>
> - Eric
>
>
> __
> Eric A. Booth
> Public Policy Research Institute
> Texas A&M University
> [email protected]
>
>
> On Apr 20, 2011, at 5:57 AM, King, Carina wrote:
>
>> Hi,
>>
>> I am having problems importing a file. It is stored as csv file and contains (I think, as I haven't been able to see the full data yet) over 1,400,000 records with 25 variables. So far I have tried the following:
>>
>> Opening it with insheet using "D:\datasets\mr_final.csv" (not enough memory), so and I increased my memory (which is now set to 10,000) and it's still not enough.
>>
>> I tried opening it in excel and wordpad so I could separate it out into smaller chunks, but it crashes the programmes when I try that.
>>
>> I attempted to use infile and specify "in 1/500000" but it comes with the error message 'file does not contain dictionary'
>>
>> So, is there a way to make a dictionary? Or change the file type so it takes less space? Or use the 'in' option with insheet? Or any other ideas?? I'm totally stuck! Thanks!
>>
>> Kind Regards,
>> Carina
>>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/