Re: st: RE: Need help with Programming/ running a forvalues loop inside foreach loop

[Sergiy Radyakin <serjradyakin@gmail.com>]

Michael, you missed the -clear- command in my previous reply.
Either add command clear before command use, or add option clear to command use.
Currently Stata resists, because it detects that you will lose the
result of your 10 appends of the first variable iteration. Since you
don't need this appended file anymore, just flush it with 'clear'.
Joe also mentioned that you need the word using between the list of
variables and the file name in the use command. Joe is right, you do
need it.
Best, Sergiy Radyakin


On Tue, Nov 5, 2013 at 7:04 PM, Michael Stewart
<michaelstewartresearch@gmail.com> wrote:
> Dear Sergiy,
>
> Thank you very much for your response.
>
> Just to clarify your suggestions, the new loop looks a follows .Is this right??
> The loop does well for var-1 (across all years) but after the cycle is
> completed, it fails for second variable (var-2) and gives a error
> saying "no; data in memory would be lost r(4);"
>
> ***********************************************
> clear
> macro drop _all
> global data "Z:\data"
> foreach outcome in  var1 var2 var3 {
> local interesting "`outcome' var5 var6 var7"
> use `interesting' "$data\dataset_1"
> forvalues i=2/10{
> append using "$data\dataset_`i'",keep(`interesting')
> }
> svyset [pweight= var5],strata  (var6 ) psu ( var7 )
> svy:mean `outcome'
> }
> ***************************************************
> Do you have any recommendations to make the loop run for all variables
> i.e., var1, var2 and var3  across all years .
>
> Thank you very much for your time.
>
> On Tue, Nov 5, 2013 at 4:34 PM, Joe Canner <jcanner1@jhmi.edu> wrote:
>> Mike,
>>
>> I don't see anything particularly wrong with the -forvalues- loop, but I think the -use- statement should also have a -using-:
>>
>> use `x' var5 var6 var7 using "$data\dataset_1",clear
>>
>> That aside, what specific symptoms are you seeing when you do the -forvalues- loop?  Are you getting errors?  What is in memory when it finishes?
>>
>> Regards,
>> Joe Canner
>> Johns Hopkins University School of Medicine
>>
>> -----Original Message-----
>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Michael Stewart
>> Sent: Tuesday, November 05, 2013 3:42 PM
>> To: statalist
>> Subject: st: Need help with Programming/ running a forvalues loop inside foreach loop
>>
>> Dear Statalist member,
>>
>> I am writing to seek help with stata programming and I am dealing with
>> rather large datasets and am resorting to loading relevant variables
>> of the dataset for analysis secondary to memory constraint.
>>
>> I especialy need help running a  forvalues loop inside a foreach loop .
>>
>> Details are as follows :
>> *************************
>> My data is in a folder "Z:data"
>>
>> My files are named dataset_1 to dataset_10 (denoting years 2001 till
>> 2010) and each file is in the order of  gigabytes with 40 variables
>> each .
>>
>> For my analysis I have to use (either var1 OR var2 OR var3) along with
>> (var5+var6+var7) from each dataset.
>>
>> As it is a multiyear analysis, I need to combine (either var1 OR var2
>> OR var3) along with (var5+var6+var7) from each dataset(dataset_1 to
>> dataset_10) and analyze.
>>
>>
>> My  dofiles is as follows
>>
>> *******************************
>> clear
>> macro drop _all
>> global data "Z:data"
>>
>> foreach x in  var1 var2 var3 {
>> use `x' var5 var6 var7 "$data\dataset_1",clear
>> forvalues i=2/10{
>> append using "$data\dataset_`i'",keep(`x' var5 var6 var7  )
>>
>> }
>> svyset [pweight= var5],strata  (var6 ) psu ( var7 )
>> svy:mean `x'
>>
>> }
>> *************************************************************
>>
>> QUESTION:  How can I make the forvalues loop  merge only relevant
>> variables from each dataset into one big dataset  before proceeding to
>> the next step so that I will have multi year dataset for analysis at
>> subsequent steps
>>
>> To further clarify the question:
>>
>> After the foreach loop  takes var1 + var 5-7 from dataset_1; I would
>> like the forvalues loop to take var1 + var 5-7 from dataset_2/10 ,
>> merge them and procedd further statistical analysis.
>>
>> In next cycle, the foreach loop will take var2 + var 5-7 from
>> dataset_1; I would like the forvalues loop to take var2 + var 5-7 from
>> dataset_2/10 , merge them and procedd further statistical analysis.
>>
>> But with my loop, the forvalues loop is not merging multipe
>> files(dataset_2 to dataset_10) before proceeding for furtehr analysis.
>>
>> Any suggestions/recommendations are welcome
>>
>> Thank you very much for your time.
>>
>> --
>> Thank you ,
>> Yours Sincerely,
>> Mike.
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>
>
>
> --
> Thank you ,
> Yours Sincerely,
> Mike.
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/