Re: st: GMM with 93+ regressors produces error

[Stas Kolenikov <skolenik@gmail.com>]

Brian,

this is a pretty peculiar case of a deeply hidden limit that does not
produce an informative error message. Can you please discuss this
situation in the help for -gmm- or in the manual in more detail? Can
you show the syntax you used for parsing Julian's expression, so that
others can reproduce it in order to judge whether they have problems
with the length of an expression? Or did you just pick the long
expression from the trace? As I said, I encountered some pretty long
-gmm- expressions in my work, and want to find out how stable my code
would be for longer lists of regressors or instruments, preferably
before sending the syntax to -gmm- only to find it breaking down.

On Sat, Feb 26, 2011 at 5:51 PM, Brian P. Poi <brian@poiholdings.com> wrote:
>
> On 2/26/2011 4:08 PM, Julian Reif wrote:
>>
>> Is it possible to run -gmm- with more than 92 regressors?  I encounter the
>> error "could not evaluate equation 1" when I try to do so.  The following
>> logit example, run on Stata 11.1, reproduces the error:
>>
>> -------------------
>> -------------------
>> clear all
>> set seed 38
>> set obs 93
>> gen id =  _n
>> expand 10
>> gen lhs = uniform()<  0.5
>> xi I.id
>>
>> * These two commands should produce the same coefficient estimates. -gmm-
>> produces an error here though.
>> logit lhs _I*
>> gmm (lhs - exp({xb: _I*}+{b0})/(1+exp({xb:}+{b0}))), instruments( _I*)
>> onestep derivative(/xb = -1*( exp({xb:}+{b0}) / ((1+exp({xb:}+{b0}))^2) ))
>>     derivative(/b0 = -1*( exp({xb:}+{b0}) / ((1+exp({xb:}+{b0}))^2) ))
>>
>> * Dropping one regressor makes error message go away
>> drop if id==93
>> drop _Iid_93
>> logit lhs _I*
>> gmm (lhs - exp({xb: _I*}+{b0})/(1+exp({xb:}+{b0}))), instruments( _I*)
>> onestep derivative(/xb = -1*( exp({xb:}+{b0}) / ((1+exp({xb:}+{b0}))^2) ))
>>     derivative(/b0 = -1*( exp({xb:}+{b0}) / ((1+exp({xb:}+{b0}))^2) ))
>>
>>
>>
>> Julian
>> *
>
> There is a limit to how long an expression passed to -generate- or -replace-
> can be, and that is what's causing the error message here. When you specify
> a substitutable expression with -gmm-, it substitutes your parameter names
> with the name of a temporary matrix and the appropriate column index.  In
> your case, -gmm- is replacing your substitutable expression with something
> like
>
> __00002X = lhs - exp( (__000006[1,1] *_Iid_2 + __000006[1,2] *_Iid_3 +
> __000006[1,3] *_Iid_4 + __000006[1,4] *_Iid_5 + __000006[1,5] *_Iid_6 +
> __000006[1,6] *_Iid_7 + __000006[1,7] *_Iid_8 ...
>
> where "..." is rather long in your case, __00002X happens to be the
> temporary variable -gmm- is using to hold the residuals and __000006 is the
> temporary matrix of parameters.
>
> When I take the exact expression -gmm- is using here and try and pass it to
> -replace-, I get
>
> .
> . replace k = `express2'
> expression too long
> r(130);
>
> In terms of -gmm-, the only solution is to write a function evaluator
> program instead.
>
>
>   -- Brian Poi
>   -- brian@poiholdings.com


-- 
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/