Re: st: Need help with working with multiple files

[Michael Stewart <michaelstewartresearch@gmail.com>]

HI Nick
Sorry to keep bothering you as I wanted to clarify the last few
sentences of your reply "save the file within
the loop _or_ otherwise use the name you want directly."

Here in my situation, the code would be

clear
local ds1 c\data\top/dataset
local ds2  c\data\date/news/dataset
...
...
..
local ds13  c\data\data/datedataset-
local all ds1 ....ds13
foreach x in `all'{
use ``x'',clear
datamagement tasks   / contains foreaach loop
datamagement tasks  / contains foreaach loop
datamagement tasks  / contains foreaach loop

save ?????
}

Here while saving the dataset , when I use  save ``x''_new,replace-->
this saves the dataset as _new and we get one dataset instead of the
13 dataset that we started with

According to your recommendation "otherwise use the name you want
directly", how can I fit 13 names here in the loop .
Am I missing some  important concept here?? Please help me with  the
problem and understand the concept.

Thanks
MIke


On Tue, May 28, 2013 at 3:40 AM, Nick Cox <njcoxstata@gmail.com> wrote:
> If you refer to a macro that doesn't exist Stata doesn't regard that
> as an error: it just substitutes nothing, or equivalently ignores the
> reference.
>
> Nested macro references don't affect this principle and indeed must be
> consistent with it. So if  local macro x does not exist as a string to
> be used neither do any of
>
> `x'
> ``x''
> ```x''''
>
> and so forth. You could say that they define empty strings if you
> like, but the effect is the same: a non-existent string and an empty
> string look identical.
>
> From what you say that is precisely what you did. It was the -foreach-
> loop that defined x as what the loop used -- and (an extra principle
> here) that macro is not defined outside the loop, and indeed is doubly
> local, to the loop as well as in the usual sense.
>
> The user manual [U] contains a basic introduction to macros in
>
> [U]     Chapter 18  . . . . . . . . . . . . . . . . . . . .  Programming Stata
>         (help program, macro, quotes, syntax, examplehelpfile)
>
> From your earlier posts it seems that you need to save the file within
> the loop _or_ otherwise use the name you want directly.
>
> Nick
> njcoxstata@gmail.com
>
> On 28 May 2013 06:00, Michael Stewart <michaelstewartresearch@gmail.com> wrote:
>> Dear Steve and Nick ,
>> I am having problem with saving the files
>> I am using ``x'' and the dofile runs through multiple dofiles like a charm
>> However, after completetion of the datamanagement tasks, the  each
>> file is being stored as "_new" in the stata working directory.
>> And the interesting part is that  this happens only if the individual
>> tasks have foreach loop in it. If the individual tasks dont have
>> foreach loop in it, then the files are saved in respective folders
>> i.e., save ``x''_new, replace --> works
>> I am not sure is there is a way to circumvent this problem
>> Thanks
>> Mike
>>
>>
>> On Mon, May 27, 2013 at 10:46 PM, Michael Stewart
>> <michaelstewartresearch@gmail.com> wrote:
>>> HI ,
>>> First of all, thank you very much Steve and Nick.It works like a charm
>>> and I can automate stata to replicate many tasks
>>> I work on Windows .Using ``x'' works.I havent tested `x'.But it
>>> Works.Thats great .
>>> Thank you Steve and Nick
>>> Thank you ,
>>> Yours Sincerely,
>>> Mike.
>>>
>>>
>>> On Mon, May 27, 2013 at 9:46 PM, Steve Samuels <sjsamuels@gmail.com> wrote:
>>>>
>>>> Well, I was concerned about that, but the code worked for me in
>>>> Mac OS X, Here was my test do file:
>>>>
>>>>
>>>> **********************************
>>>> local ds1 "~Downloads/testsim01"
>>>> local ds2 "~Downloads/dtest01"
>>>>
>>>> local all ds1 ds2
>>>> di `"`all'"'
>>>> foreach x in `all' {
>>>> use `x', clear
>>>> gen nv =1
>>>> save `x'_new, replace
>>>> }
>>>> ********************************
>>>>
>>>> Steve
>>>>
>>>> On May 27, 2013, at 8:50 PM, Nick Cox wrote:
>>>>
>>>> This won't work, as when Stata hits the loop
>>>>
>>>> foreach x in `all'
>>>>
>>>> it will then see
>>>>
>>>> foreach x in ds1 ds2
>>>>
>>>> and will fail at
>>>>
>>>> use ds1, clear
>>>>
>>>> There are two levels of macro referencing in Steve's code, but only
>>>> one level going backwards.
>>>>
>>>> use ``x'', clear
>>>>
>>>> is one way to fix it.
>>>>
>>>> Nick
>>>> njcoxstata@gmail.com
>>>>
>>>>
>>>> On 28 May 2013 01:23, Steve Samuels <sjsamuels@gmail.com> wrote:
>>>>>
>>>>> Michael Steward (michaelstewartresearch@gmail.com):
>>>>>
>>>>> You can loop, as Nick suggests. Put the data management tasks in
>>>>> external do files, e.g. tasks1.do, tasks2.do, tasks3.do. Then something
>>>>> like the following should work (not tested in Windows). See the help for
>>>>> -include-. The big problem I anticipate is the mounds of output that
>>>>> this could generate.
>>>>>
>>>>>
>>>>> *********************************
>>>>> local ds1 "c/data/dataset-1"
>>>>> local ds2 "c/data/newdata/dataset-2" // etc.
>>>>>
>>>>> local all ds1 ds2
>>>>>
>>>>> foreach x in `all' {
>>>>>   use `x', clear
>>>>>
>>>>>   include tasks1
>>>>>   include tasks2
>>>>>   include tasks3
>>>>>
>>>>>   save `x'_new, replace
>>>>> }
>>>>> *********************************
>>>>>
>>>>> Steve
>>>>>
>>>>>
>>>>> On May 27, 2013, at 3:19 PM, Michael Stewart wrote:
>>>>>
>>>>> Hi
>>>>> I am trying to find a way to work multiple files successively.
>>>>> I have multiple large files(really large) placed in different directories
>>>>> I am trying to find a way to load each file into stata, complete bunch
>>>>> of same datamagement tasks,create  a new file from it and places them
>>>>> in a common folder
>>>>>
>>>>> I could use a dofile and do something like
>>>>>
>>>>> clear
>>>>> use c\data\dataset-1
>>>>> datamagement tasks
>>>>> datamagement tasks
>>>>> datamagement tasks
>>>>> save c\data\dataset-1_new
>>>>> clear
>>>>> use c\data\newdata\dataset-2
>>>>> datamagement tasks
>>>>> datamagement tasks
>>>>> datamagement tasks
>>>>> save c\data\newdata\dataset-2_new
>>>>> clear
>>>>>
>>>>> But is there an elegant and simpler way to do it given that I have
>>>>> nearly 15 files to do data management tasks.
>>>>>
>>>>> Can we use foreach loop / forvalues loop or anything like that .Please suggest
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/



-- 
Thank you ,
Yours Sincerely,
Mike.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/