RE: st: RE: ivreg2 warning, even after "partialling out"

["Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>]

Jin,

> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> statalist@hsphsun2.harvard.edu] On Behalf Of Jin-Hyun Bae
> Sent: 14 November 2013 17:19
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: RE: ivreg2 warning, even after "partialling out"
> 
> I just noticed that the $exogctrl results are no longer reported.
> Would it be "correct" to use the ones produced in the other regression
> without partialling out of $exogctrl,

Should be OK if you don't partial out $exogctrl.  There are only 15 of them.

--Mark

> or is there anything else I should be doing
> that I am missing?
> 
> Jin
> 
> 
> On 14 November 2013 12:01, Jin-Hyun Bae <jinhyun.bae@stern.nyu.edu>
> wrote:
> > Mark,
> >
> > Thank you for the clarification. I ran the code that you had in mind
> > and it now produces results without warnings.
> >
> > I have 15 variables in $exogctrl and original _I* has a total of 182.
> > Hardly surprising that stata "gave up" when there are about 2300
> > observations and I previously ran the code with thousands of
> > additional _I variables!
> >
> > Thank you again,
> > Jin
> >
> >
> > On 14 November 2013 11:41, Schaffer, Mark E <M.E.Schaffer@hw.ac.uk>
> wrote:
> >> Jin,
> >>
> >> It sounds like you have too many explicit regressors for cluster, and
> maybe too many in general - if you have many hundreds of explicit
> regressors,  Stata will have to invert a very large matrix, and that is inviting
> numerical accuracy problems.
> >>
> >> A few thoughts:
> >>
> >> Try partialling out $exogctrl AND _I* in the same regression, i.e.,
> partial($exogctrl _I*).  This is what I originally had in mind.
> >>
> >> How many variables are in $exogctrl?  How many are in _I*?
> >>
> >> What is matsize set to?  You can find out with
> >>
> >>         query memory
> >>
> >> in Stata.
> >>
> >> --Mark
> >>
> >>> -----Original Message-----
> >>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> >>> statalist@hsphsun2.harvard.edu] On Behalf Of Jin-Hyun Bae
> >>> Sent: 14 November 2013 14:19
> >>> To: statalist@hsphsun2.harvard.edu
> >>> Subject: Re: st: RE: ivreg2 warning, even after "partialling out"
> >>>
> >>> Hello Mark,
> >>>
> >>> Thank you for your suggestion. I tried to partial out the full list
> >>> of controls and also a subset excluding binary controls, but I get
> >>> the following error.
> >>>
> >>> ivreg2 selfemp1 cultdist $exogctrl _I*, cluster (nat cisstate)
> >>> partial(_I*) matsize too small r(908);
> >>>
> >>> Also, since you mentioned that it would be difficult to see where
> >>> the problem lies without seeing the results, I would like to respond
> >>> to that. Unfortunately, the data restrictions I have prevent me from
> >>> sending out the raw results. Could you tell me what you would like
> >>> to know, or what I should be looking out for?
> >>>
> >>> Thank you again,
> >>> Jin
> >>>
> >>> Date: Thu, 14 Nov 2013 00:23:42 +0000
> >>> From: "Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>
> >>> Subject: st: RE: ivreg2 warning, even after "partialling out"
> >>>
> >>> Jin,
> >>>
> >>> > -----Original Message-----
> >>> > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> >>> > statalist@hsphsun2.harvard.edu] On Behalf Of Jin-Hyun Bae
> >>> > Sent: 13 November 2013 23:20
> >>> > To: statalist@hsphsun2.harvard.edu
> >>> > Subject: st: ivreg2 warning, even after "partialling out"
> >>> >
> >>> > Hello!
> >>> >
> >>> > I'm having problems nailing down the cause of a warning message
> >>> > when
> >>> using
> >>> > ivreg2. I am running OLS with two-way clustering (nat, cisstate)
> >>> > and have
> >>> added
> >>> > dummies for these variables.
> >>> >
> >>> > selfemp1 is a binary variable.
> >>> > cultdist is a continuous variable.
> >>> > _I* is a list of dummies based on nat and cisstate. I created
> >>> > these based on
> >>> the
> >>> > example code in the ivreg2 help file.
> >>> >
> >>> > When I run
> >>> > ivreg2 selfemp1 cultdist _I*, cluster (nat cisstate) partial(_I*)
> >>> > I don't get a warning.
> >>> >
> >>> > But when I add a list of exogenous controls ($exogctrl), I get the
> >>> > following
> >>> > "Warning: estimated covariance matrix of moment conditions not of
> >>> > full
> >>> rank.
> >>> > model tests should be interpreted with caution."
> >>> >
> >>> > This is the exact code I used.
> >>> > ivreg2 selfemp1 cultdist $exogctrl _I*, cluster (nat cisstate)
> >>> > partial(_I*)
> >>> >
> >>> > Also, if it helps, I get the same message when using the following code.
> >>> > ivreg2 selfemp1 cultdist $exogctrl, cluster (nat cisstate)
> >>> >
> >>> > In all cases I have 77 clusters for nat and 42 clusters for
> >>> > cisstate, and there
> >>> are
> >>> > no missing F values or std errors.
> >>> >
> >>> > I updated ivreg2 beforehand so it is up to date.
> >>> >
> >>> > Jin
> >>>
> >>> It's hard to tell for sure without seeing the output, but most
> >>> likely you are running into the
> >>> not-full-rank-because-of-not-enough-clusters
> >>> issue.
> >>>
> >>> Does the error message go away if you also partial out the
> >>> additional exogenous controls in $exogctrl?
> >>>
> >>> - --Mark
> >>>
> >>> > *
> >>> > *   For searches and help try:
> >>> > *   http://www.stata.com/help.cgi?search
> >>> > *   http://www.stata.com/support/faqs/resources/statalist-faq/
> >>> > *   http://www.ats.ucla.edu/stat/stata/
> >>>
> >>>
> >>> - -----
> >>> Sunday Times Scottish University of the Year 2011-2013 Top in the UK
> >>> for student experience Fourth university in the UK and top in
> >>> Scotland (National Student Survey
> >>> 2012)
> >>>
> >>> We invite research leaders and ambitious early career researchers to
> >>> join us in leading and driving research in key inter-disciplinary themes.
> >>> Please see www.hw.ac.uk/researchleaders for further information and
> >>> how to apply.
> >>>
> >>> Heriot-Watt University is a Scottish charity registered under
> >>> charity number SC000278.
> >>> *
> >>> *   For searches and help try:
> >>> *   http://www.stata.com/help.cgi?search
> >>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> >>> *   http://www.ats.ucla.edu/stat/stata/
> >>
> >>
> >> -----
> >> Sunday Times Scottish University of the Year 2011-2013 Top in the UK
> >> for student experience Fourth university in the UK and top in
> >> Scotland (National Student Survey 2012)
> >>
> >>
> >> We invite research leaders and ambitious early career researchers to
> >> join us in leading and driving research in key inter-disciplinary themes.
> >> Please see www.hw.ac.uk/researchleaders for further information and
> >> how to apply.
> >>
> >> Heriot-Watt University is a Scottish charity registered under charity
> >> number SC000278.
> >>
> >>
> >> *
> >> *   For searches and help try:
> >> *   http://www.stata.com/help.cgi?search
> >> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> >> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/


----- 
Sunday Times Scottish University of the Year 2011-2013
Top in the UK for student experience
Fourth university in the UK and top in Scotland (National Student Survey 2012)

We invite research leaders and ambitious early career researchers to 
join us in leading and driving research in key inter-disciplinary themes. 
Please see www.hw.ac.uk/researchleaders for further information and how
to apply.

Heriot-Watt University is a Scottish charity
registered under charity number SC000278.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/