Re: st: Foreach loop, panel data, and residuals

[Nick Cox <njcoxstata@gmail.com>]

Ulrich is correct. Note here that

forv i = 1/`=r(max)' {

and

forv i = 1/`r(max)' {

work equally well.

-foreach- tries before working through the loop to construct a list of
all the elements you want to cycle over, namely

1 2 3 4 ... 36221

where naturally I left out almost every element. In your case that is
too long. The list

1 2 3 4 ... 758510

would be even longer and so could not possibly solve your problem,
quite apart from the detail that you want to cycle over groups, not
individuals, so -group- would not take any values 36222 ... 758510.

-forval- does not try to construct a list in advance and is free of this limit.

Nick
njcoxstata@gmail.com


On 19 December 2013 11:57, Ulrich Kohler <ukohler@uni-potsdam.de> wrote:
> Use -forvalues- instead of -foreach-, i.e.
>
> . forv i=1/`=r(max)' {
> ...
> }
>
>
> Am 19.12.2013 12:54, schrieb Adrian Stork:
>
>> Dear Nick
>> deal all
>>
>> So, as Nick suggested I tried to implement the loop using method one
>> from the link above. So the code I ended up with looks like this:
>>
>> *******************************************
>> egen group = group(cusip date2)
>> gen residual = .
>>
>> su group, meanonly
>>
>> foreach i of numlist 1/`r(max)' {
>>         regress ex_ret_daily mktrf smb hml if group == `i'
>>         predict temp, residuals
>>         replace residual=temp if group == `i'
>>         drop temp
>> }
>> *******************************************
>>
>> Although, this code makes sense at least to me, it does not work and I
>> get the error message:
>> invalid numlist has too many elements
>> r(123);
>>
>> In total I have 758,510 obs and r(max) is 36,221.
>> I also used r(N) instead of r(max) and still the same error message.
>> I also extracted a subsample with merely 3677 obs and 2 cusips (i.e.
>> two groups, i.e. r(max)=2)
>>
>> Does anyone have an idea how I can make this loop work based on the
>> description I previously gave or where my mistake is?
>> Would very much appreciate it.
>>
>> Best,
>> A
>>
>>
>>
>>
>>
>>
>> 2013/12/17 Nick Cox <njcoxstata@gmail.com>:
>>>
>>> Your -foreach- command cycles over a list containing precisely one
>>> element, the variable -group-.
>>>
>>> foreach i of var group {
>>>   regress ex_ret_daily mktrf smb hml if group == `i'
>>>   predict temp, residuals
>>>   replace residual=temp if group == `i'
>>>   drop temp
>>> }
>>>
>>> It is thus on all fours with
>>>
>>>   regress ex_ret_daily mktrf smb hml if group == group
>>>   predict temp, residuals
>>>   replace residual=temp if group == group
>>>   drop temp
>>>
>>> which carries out the regression for every observation as -if group ==
>>> group- selects all the data,
>>>
>>> Your intent is to cycle over the distinct values of -group-, which is
>>> quite a different matter. See Method 1 in
>>>
>>> http://www.stata.com/support/faqs/data-management/try-all-values-with-foreach/index.html
>>>
>>> Nick
>>> njcoxstata@gmail.com
>>>
>>> On 17 December 2013 16:36, Adrian Stork <storkadrian@googlemail.com>
>>> wrote:
>>>
>>>> I'm close to the solution I'm looking for but double-checking shows
>>>> some deviations that I can't figure out.
>>>> So I got a panel dataset defined by cusip and date which looks like
>>>> this:
>>>>
>>>> cusip                 date2                 date
>>>> ex_ret_daily               mktrf             smb              hml
>>>> 90000000         1990m1            01Jan1990              0.10
>>>>                   1.5               0.3                0.2
>>>> 90000000         1990m1            02Jan1990              0.40
>>>>                   0.7                0.6               0.7
>>>> ....                    ....                     ....
>>>>          ...                          ...                ....
>>>>      ....
>>>> 90000004         1983m1            01Jan1983              0.14
>>>>                   1.5               0.3                0.9
>>>> 90000007         1983m1            02Jan1983              0.45
>>>>                   1.7                0.6               0.3
>>>> etc.
>>>>
>>>> Now I want to retrieve the residuals from the regression:
>>>> ex_ret_daily= beta*mktrf + smb + hml
>>>> but for each cusip and month, that is, using the approx 25 daily
>>>> returns from a month of a cusip as the subsample each regression
>>>> should run on.
>>>> I wrote a code that works and the results even look reasonable, which
>>>> looks as follows:
>>>>
>>>> ***
>>>> egen group = group(cusip date2)
>>>> gen residual = .
>>>> foreach i of var group {
>>>>   regress ex_ret_daily mktrf smb hml if group == `i'
>>>>   predict temp, residuals
>>>>   replace residual=temp if group == `i'
>>>>   drop temp
>>>> }
>>>> ***
>>>>
>>>> However, when I compare the residuals from my code and the residuals
>>>> from a subsample of one cusip of one specific month, they are not
>>>> identical, and I don't know where I could've made a mistake?
>>>
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>> *   http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/