Re: st: Appending multiple large datasets

[Sergiy Radyakin <serjradyakin@gmail.com>]

"Windows informs me Stata has stopped working"
this does not mean that Stata has stopped working. This means that
Stata is busy, and failed to respond to Windows' request after a
certain period. Usually this occurs if the program is not redrawing
it's window for a while.

You may want to :
1) extract a few lines from each of your original files into a
separate smaller copies and try appending them for diagnostics;
2) try your script putting the same filename twice (either first or
second) and see whether any or both fail;
3) insert -describe- command after each -insheet- and tell us the results.

Best, Sergiy


On Fri, Jun 7, 2013 at 3:52 PM, Ryan Stevens <stevrl7@gmail.com> wrote:
> No error messages, Windows informs me Stata has stopped working, so it
> could be a problem with Windows. The insheet commands individually
> work, I have checked this. I'll not set the memory myself. And yes,
> Excel reads them fine. I can try importing as XLS to see if that might
> change something. As a side note, I've been able to workaround this by
> merging the files since none of the data is duplicated between my
> files. This does work, but I'm still wanting to figure out what
> exactly is wrong.
>
> Ryan
>
> On Fri, Jun 7, 2013 at 3:44 PM, Sergiy Radyakin <serjradyakin@gmail.com> wrote:
>> So what exactly did not work?
>> Which error message are you getting?And after which line?
>> Insert command count after each insheet. Check how many observations
>> got imported.
>> Do not set memory in Stata 12. It can do it better then the user.
>> Are your CSV files really "comma separated values"? Does Excel read
>> them OK? Try saving from Excel and then importing XLS file into Stata
>> 12.0.
>> Sergiy.
>>
>> On Fri, Jun 7, 2013 at 1:57 PM, Ryan Stevens <stevrl7@gmail.com> wrote:
>>> Hi all,
>>>
>>> I'm experiencing a problem appending 2 datasets containing roughly 1
>>> million observations each. I'm currently running Stata 12.0. My code
>>> can import each dataset fine, but it always gets hung up when I try to
>>> append the two datasets. My code using temporary files is below and is
>>> very simple. I've tried creating physical .dta files and saving them
>>> and that did not work. I can assure you my directories work fine, so
>>> it's no issues with names of my files. I'm wondering if there are any
>>> thoughts on where this problem could lie.
>>>
>>> clear
>>> capture log close
>>> set memory 100m
>>> set more off
>>> macro drop _all
>>>
>>> *Import data
>>> insheet using "consump1.csv"
>>> tempfile master_file
>>> save "`master_file'"
>>>
>>> clear
>>> insheet using "consump2.csv"
>>> append using "`master_file'"
>>>
>>>
>>>
>>> Thanks for any assistance,
>>> Ryan
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>> *   http://www.ats.ucla.edu/stat/stata/
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/