Re: st: making code faster

[Nick Cox <njcoxstata@gmail.com>]

Study this example:

clear
* below not in Stata 12
* set memory 100M
sysuse auto
expand 10000

set rmsg on

gen touse = !missing(mpg, weight, foreign, rep78)

bysort touse foreign rep78 : replace touse = 0 if _N == 1
egen group = group(foreign rep78) if touse

su group, meanonly
local max = r(max)

gen residuals = .

qui forval i = 1/`max' {

	reg mpg weight if group == `i'
	predict temp, residuals
	replace residuals=temp if group == `i'
	drop temp

}

gen residuals2 = .
sort group
gen long obs = _n

qui forval i = 1/`max' {
      su obs if group == `i', meanonly
	local min = r(min)
	local max = r(max)
	reg mpg weight in `min'/`max'
	predict temp, residuals
	replace residuals2 =temp in `min'/`max'
	drop temp
}

The second way of doing it is a bit faster, but you have to set it up.
Note that if the issue is that your dataset is too large for Stata to
fit into memory, the only real remedy is to buy more memory.

Nick

On Mon, May 14, 2012 at 3:09 PM, Rudy Hover <rudy.hover@gmail.com> wrote:
>> I have a lot of regressions to run (on stock returns of firms over a
>> periof of 7 years) in order to retrieve the residuals.
>>
>> I am using the code
>>
>>
>> gen touse = !missing(excessreturn, marketreturn, firmno, year)
>>
>> bysort touse firmno year : replace touse = 0 if _N == 1
>>
>> egen group = group(firmno year) if touse
>>
>> su group
>>
>> local max = r(max)
>>
>> gen residuals = .
>>
>> qui forval i = 1/`max' {
>>
>>  reg excessreturn marketreturn if group == `i'
>>
>>  predict temp, residuals
>>
>>  replace residuals=temp if group == `i'
>>
>>  drop temp
>>
>> }
>>
>>
>>
>> How could I make it any faster using -in- instead of -if- ?
>>
>> It takes hours now.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/