RE: st: Routine for merging many text-files

[Simon Falck <simon.falck@abe.kth.se>]

Thanks Mani for this. You are right, as Nick suggested, this is a job for -append-, and it works fine. I will look into your suggestion on how to create a batch file before import to Stata.

Simon


-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Mani Subramani
Sent: den 20 mars 2013 09:08
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: Routine for merging many text-files

If I understand your problem right, you are trying to combine about
100 text files with identical structure for analysis. As Nick suggested, the 'append' command should do the job within Stata However, this can be very simply accomplished in Linux or Windows before submitting the job to Stata. If you are using a Windows machine, the following command from the command line (within the directory where you have the files):
copy /a *.txt combined.txt
will combine the whole batch of files into one text file which you can import and append to your dataset.



On Fri, Mar 15, 2013 at 6:02 AM, Simon Falck <simon.falck@abe.kth.se> wrote:
> Hi,
>
> I have about 100 text files that I want to merge in to a longitudinal datab= ase and wonder if there is a routine that I can apply to ease this work.
>
> The text-files have different periods but the same datastructure, 
> which is = a wide format that looks something like this,
>
> Id  y1  y2  y3
> 1  10  30  40
> 2  11  31  41
> ...and so on..
>
> If I would apply the standard procedure to create the database the 
> script w= ould look something like this for two text-files, (as an 
> example but would = look the same for the 100 text-files)
>
> *Insert dataset 1 from textfile and save as .dta insheet using "C:\User\dataset1", tab reshape long y, i(id) j(year) rename y var1 save "C:\User\dataset1.dta"
>
> *Insert dataset 2 from textfile and save as .dta insheet using "C:\User\dataset2", tab reshape long y, i(id) j(year) rename y var2 save "C:\User\dataset2.dta"
>
> *Merge dataset 1 and 2 to key-file containing joint id=B4s.
> use "C:\User\id.dta"
> merge 1:m id year using "C:\User\dataset1.dta"
> drop _merge
> merge 1:m id year using "C:\User\dataset2.dta"
> drop _merge
>
> The resulting database would look something like this,
>
> Id  year  var1 var2
> 1     1         10   100
> 1     2         15    75
> 1     3        20    65
> 2     1       11    112
> 2     2       17     80
> 3     1       36    110
> ...and so on..
>
> Since I have about 100 text-files that (1) needs to be converted into 
> .dta,=
>  (2) reshaped into a long-format, (3) rename variables, and (4) merged into=  a joint database, I wonder if someone know how I could write a routine to = ease this work which otherwise is repetitive and result in a very large .do=  file.
>
> Thanks in advance
> Simon
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/



--
Mani Subramani, msubramani@umn.edu
Information and Decision Sciences Department Carlson School of Management, University of Minnesota Minneapolis, MN 55455.
612-624-3522
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/