Re: st: extracting of data

[Nick Cox <njcoxstata@gmail.com>]

Sorry, but this does not add any precise information to what you have
already given. Hence I can't add to my previous post except that what
I have suggested seems simpler than what you are trying. You know
about looping over levels, you know about -infile- and -save-, so all
you have do is to -append- files. If Statalist is not working out for
you, you need to get help elsewhere e.g. from a local Stata expert or
Stata tech support.

Nick

On Tue, Jan 22, 2013 at 4:54 AM, Prakash Singh <prakashbhu@gmail.com> wrote:
> Nick first of all I should tell you I am not use to with programming,
> what I have written is through reading the stata post and using my
> small capability, so there is huge chances of error for that reason I
> seek help.
>
> Now, I would try to explain you the logic of my code and loop.
> As you know the raw data is structured so that I need to extract 81
> (9*9) data files and then append set of 9 files into one. This takes
> lot of space in the system, so I thought of writing code where I can
> extract the data but finally I should have only appended data files.
> I just wanted to get rid off 81 data file space consumption.
>
> One more bit of information, it seems that the code I have written is
> only reading the first dct file.
>
> I hope this will help you to understand and help me come out of this.
> I tried to work out the suggestion of Daniel and yours but I failed. I
> am still trying to work it out.
>
> Regards
> Prakash
>
> On Tue, Jan 22, 2013 at 12:00 AM, Nick Cox <njcoxstata@gmail.com> wrote:
>> You are resurrecting a thread from January 1, but your code raises
>> questions very similar to those in a thread you started today,
>> principally that
>>
>> 1. The logic of your loop is hard to follow.
>>
>> 2. Using -capture- prevents you (and us) from seeing error messages
>> that might be informative.
>>
>> As I understand it, you have 9 files for levels 1...9 and within your loop you
>>
>>          infile using D:\TEAM_PC\level`l'\level`l'.dct, clear
>>          capture append using `mydata'
>>          save `mydata', replace
>>          keep if level==`l'
>>          save "D:\TEAM_PC\level`l'_list.dta", replace
>>
>> However, whatever the -append- and -save- do is cancelled out by the
>> -keep-, so you might as well go
>>
>>          infile using D:\TEAM_PC\level`l'\level`l'.dct, clear
>>          save "D:\TEAM_PC\level`l'_list.dta"
>>
>> So that will give you 9 files and you can -append- them in one
>> command; no loop is needed. See the on-line help for -append-
>>
>> http://www.stata.com/help.cgi?append
>>
>> (If your Stata version does not support this multiple file syntax,
>> then as advised in the FAQ you should be telling us what version you
>> have.)
>>
>> I can't tell you what is going on with your `mydata'. As Daniel
>> pointed out earlier today, it is unclear why you are using a temporary
>> file here, but that's another detail.
>>
>> Nick
>>
>> On Mon, Jan 21, 2013 at 6:15 PM, Prakash Singh <prakashbhu@gmail.com> wrote:
>>> Nick
>>> Though I managed to write the code and run it but I realised today
>>> that for each level stata is extracting data from only one txt file.
>>> which means that only one dct file is being read.
>>>
>>> here is the revised code
>>>
>>> clear
>>> set more off
>>> set mem 500m
>>> tempfile mydata
>>> forv l=1/9 {
>>>         infile using D:\TEAM_PC\level`l'\level`l'.dct, clear
>>>         capture append using `mydata'
>>>         save `mydata', replace
>>>         keep if level==`l'
>>>         save "D:\TEAM_PC\level`l'_list.dta", replace
>>> }
>>>
>>>
>>> Regards
>>> Prakash
>>>
>>> On Tue, Jan 1, 2013 at 9:32 PM, Prakash Singh <prakashbhu@gmail.com> wrote:
>>>> Thanks Nick
>>>> I got it finally
>>>>
>>>> Best wishes to all stata listers for 2013
>>>>
>>>> Regards
>>>> Prakash
>>>>
>>>> On Mon, Dec 31, 2012 at 11:10 PM, Nick Cox <njcoxstata@gmail.com> wrote:
>>>>> I don't understand what is holding you up. Your code shows that you
>>>>> are familiar with -foreach- and -forval-. So you need to write another
>>>>> big loop using that construct.
>>>>>
>>>>> Nick
>>>>>
>>>>> On Mon, Dec 31, 2012 at 4:37 PM, Prakash Singh <prakashbhu@gmail.com> wrote:
>>>>>> Nick, I thanked you for that
>>>>>>
>>>>>> I mentioned in the first mail that the structure of the data is such
>>>>>> that to have final data I need to extract 9 (using 9 dct file for each
>>>>>> level) different level files then some of them I need to reshape and
>>>>>> then merge them.
>>>>>>
>>>>>>
>>>>>> Hope, I have made my query more clear to you.
>>>>>>
>>>>>> regards
>>>>>> Prakash
>>>>>> On Mon, Dec 31, 2012 at 9:48 PM, Nick Cox <njcoxstata@gmail.com> wrote:
>>>>>>> You don't comment on my replacement of 9 lines of code with 1.
>>>>>>>
>>>>>>> Your main loop looks like another -forval- or -foreach- loop but you
>>>>>>> don't explain why you want to save different levels in different
>>>>>>> datasets.
>>>>>>>
>>>>>>> Nick
>>>>>>>
>>>>>>> On Mon, Dec 31, 2012 at 2:27 PM, Prakash Singh <prakashbhu@gmail.com> wrote:
>>>>>>>> Nick sorry but that is type error.
>>>>>>>> Thanks for the suggestion anyway.
>>>>>>>>
>>>>>>>> Moreover I am looking some help on creating look so that I can repeat
>>>>>>>> the following code for each level 1 to 9.
>>>>>>>>
>>>>>>>> clear
>>>>>>>> set more off
>>>>>>>> set mem 500m
>>>>>>>> tempfile mydata
>>>>>>>> foreach l of numlist 11/19 {
>>>>>>>>    infile using D:\TEAM_PC\level`l'.dct, clear
>>>>>>>>    capture append using `mydata'
>>>>>>>>     save `mydata', replace
>>>>>>>> }
>>>>>>>> use `mydata', clear
>>>>>>>> keep if level==1
>>>>>>>> save level1_list.dta", replace
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/