Austin,
Very convincing - a nice example of how het-robust may be superior to cluster-robust for the RE estimator in some circumstances.
The behaviour of -xtreg,re- under Stata 11 for reporting SEs seems very peculiar to me:
1. The default VCE is vce(conventional).
2. Het-robust SEs are valid under a broader range of assumptions than conventional SEs, but are not available as an option and can be obtained only via version control.
3. The help file says vce(robust) and vce(cluster <clustvar>) are both options, but in fact the former automatically defaults to the latter.
4. Het-robust SEs can be superior to cluster-robust in finite samples under some circumstances (your example), but are not available as an option and can be obtained only via version control.
I can't think of another example in Stata where a valid VCE (that in some cases may be the VCE of choice) can be obtained only via version control. Seems like a bad precedent to me - what if the new version of the estimator is superior to the old version in other respects, or differs in its behaviour with respect to postestimation commands or the macros it leaves behind?
It would be much better for -xtreg,re- to revert to standard Stata behaviour and for vce(robust) to produce the usual het-robust SEs.
Just my 0.02, as they say.
--Mark
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> Austin Nichols
> Sent: 22 August 2009 14:13
> To: [email protected]
> Subject: Re: st: Stata 11 Random Effects--Std. Errors
>
> For example:
>
> clear all
> prog reclr, rclass
> version 10.1
> drawnorm x1-x10 v, n(50) clear
> forv i=1/10 {
> replace x`i'=exp(x`i')
> }
> g id=_n
> expand 20
> g e=rnormal()
> bys id: replace e=e[_n-1]/2+e/2 if _n>1
> g y=(x1+x2+x3+x4+x5)/5+v+e
> xtreg y x1-x10, cl(id) i(id) re
> test x1 x2 x3 x4 x5
> return scalar c1=r(p)<.05
> test x6 x7 x8 x9 x10
> return scalar c0=r(p)<.05
> xtreg y x1-x10, r i(id) re
> test x1 x2 x3 x4 x5
> return scalar r1=r(p)<.05
> test x6 x7 x8 x9 x10
> return scalar r0=r(p)<.05
> eret clear
> end
> simul, seed(1) rep(10000): reclr
> su
>
> This shows rejection rates for RE of 9% for a nominal 5% alpha with
> het-robust SEs but 41% with cluster-robust SEs.
>
> Here's another example, where my preferred default does not
> do so well:
>
> clear all
> prog reclr, rclass
> version 10.1
> drawnorm x1-x10 v, n(50) clear
> g id=_n
> expand 20
> forv i=1/10 {
> replace x`i'=exp(x`i'*4/5+rnormal()/5)
> }
> g e=rnormal()
> bys id: replace e=e[_n-1]*4/5+e/5 if _n>1
> g y=(x1+x2+x3+x4+x5)/5+v+e
> xtreg y x1-x10, cl(id) i(id) re
> test x1 x2 x3 x4 x5
> return scalar c1=r(p)<.05
> test x6 x7 x8 x9 x10
> return scalar c0=r(p)<.05
> xtreg y x1-x10, r i(id) re
> test x1 x2 x3 x4 x5
> return scalar r1=r(p)<.05
> test x6 x7 x8 x9 x10
> return scalar r0=r(p)<.05
> xtreg y x1-x10, i(id) re
> test x1 x2 x3 x4 x5
> return scalar o1=r(p)<.05
> test x6 x7 x8 x9 x10
> return scalar o0=r(p)<.05
> xtreg y x1-x10, cl(id) i(id) fe
> test x1 x2 x3 x4 x5
> return scalar fec1=r(p)<.05
> test x6 x7 x8 x9 x10
> return scalar fec0=r(p)<.05
> xtreg y x1-x10, i(id) fe
> test x1 x2 x3 x4 x5
> return scalar feo1=r(p)<.05
> test x6 x7 x8 x9 x10
> return scalar feo0=r(p)<.05
> eret clear
> end
> simul, seed(1) rep(10000): reclr
> su
>
>
> On Fri, Aug 21, 2009 at 11:00 AM, Austin
> Nichols<[email protected]> wrote:
> > Mark--
> > I concur that getting het-robust SEs should still be possible with
> > -xtreg, re-. I wonder, for example, about the relative
> performance in
> > smallish samples where T and N are both under 100 and there are many
> > explanatory variables. The cluster-robust SE will offer uniformly
> > better performance as N goes to infinity, I suspect, but in
> some kinds
> > of small finite samples the het-robust SE may be better, and users
> > should have that option IMHO. Unless there is strong evidence that
> > cluster-robust is always better than het-robust in RE
> models, which I
> > have not seen. On the other hand, I would make
> cluster-robust SEs the
> > default (for both FE and RE), instead of
> -vce(conventional)-, and make
> > users spell out vce(conventional) [literally] to get that inferior
> > estimator. Of course, I would also make -fe- the default
> for -xtreg-
> > in Stata 11.1, and make users spell out -re- to get that (more often
> > inferior) estimator. Why not help people make good choices with the
> > defaults? (I would also make het-robust SEs the default
> for -regress-
> > and make folks specify vce(ols) for the old default.)
> >
> > On Wed, Aug 19, 2009 at 2:19 PM, Schaffer, Mark
> E<[email protected]> wrote:
> >> This is different from the reason provided by Stock-Watson
> (2008) for
> >> FEs (see earlier in the thread).
> >>
> >> Stock and Watson show that for the fixed effects
> estimator, the usual
> >> robust VCV - i.e., robust but not cluster-robust - is not
> consistent.
> >> It's another example of the "incidental parameters"
> problem. This is
> >> why Stata 10 won't report it under any circumstances,
> including version
> >> control.
> >>
> >> My suspicion was that this might carry over to the usual
> robust VCV for
> >> the random effects estimator and that this was why it was
> dropped in
> >> Stata 11, but apparently not. Using cluster-robust with RE is
> >> apparently just following standard practice in the literature.
> >>
> >> If so, though, then I think I'd prefer to see non-cluster
> robust SEs
> >> available with the RE estimator through an option rather
> than version
> >> control.
> >>
> >> --Mark (left scratching his head and wondering why robust
> and cluster
> >> robust are both kosher for the RE estimator, only cluster-robust is
> >> kosher for the FE estimator, Stata's PA estimator allows
> only robust but
> >> not cluster-robust, and Stata's MLE estimator allows neither)
> >>
> >>
> >>> -----Original Message-----
> >>> From: [email protected]
> >>> [mailto:[email protected]] On Behalf Of DE
> >>> SOUZA Eric
> >>> Sent: 19 August 2009 18:37
> >>> To: '[email protected]'
> >>> Subject: RE: st: Stata 11 Random Effects--Std. Errors
> >>>
> >>> To add more detail:
> >>> Two of the assumptions underlying the random effects
> model are that:
> >>> 1. the unobservable effects are not correlated with the
> >>> explanatory variables 2. the residuals have the random
> >>> effects structure The second assumption is not required for
> >>> consistency.
> >>> Wooldridge's argument is: then why impose it. "With fixed T
> >>> and large N asymptotics, we lose nothing in using robust
> >>> standard errors and test statistics even if [the random
> >>> effects error structure] holds".
> >>>
> >>> Eric
> >>>
> >>>
> >>> Eric de Souza
> >>> College of Europe
> >>> Brugge (Bruges), Belgium
> >>> http://www.coleurope.eu
> >>>
> >>> -----Original Message-----
> >>> From: [email protected]
> >>> [mailto:[email protected]] On Behalf Of
> >>> David M. Drukker
> >>> Sent: 19 August 2009 19:04
> >>> To: [email protected]
> >>> Subject: RE: st: Stata 11 Random Effects--Std. Errors
> >>>
> >>> Continuing the thread about why -xtreg .., re vce(robust)-
> >>> changed its meaning between Stata 10 and Stata 11, Mark E
> >>> Schaffer <[email protected]> asked for a citation.
> >>>
> >>> Wooldridge (2002, section 10.4.2) recommends the new approach
> >>> of having
> >>> -vce(robust)- mean -vce(cluster panelvar)-.
> >>>
> >>> When we first offered the -vce(robust)- and -vce(cluster
> >>> clustvar)- on -xtreg, re-, we wanted to offer users the
> >>> flexibility to use either estimator. We had Monte Carlo
> >>> simulations showing that there are data generating processes
> >>> for which -vce(robust)- performs quite well, so we allowed
> >>> users to choose.
> >>>
> >>> Since we first offered these options, the literature has
> >>> focused on always using -vce(cluster panelvar)- to obtain a
> >>> "robust" estimator for -xtreg, re-. We made the change
> >>> because the new method is what users now expect and it offers
> >>> additional generality.
> >>>
> >>> David
> >>> [email protected]
> >>>
> >
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
--
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
