Okay that makes sense. For a second there I thought I was not
understanding the test. The different model specifications I use give
p values (from the xtoverid test) of .1 to .25. Do you think values
over say 20% make you less nervous about accepting RE results? My plan
is to report both FE and RE models, suggesting that RE results can be
considered valid given the p values.
-Steve
On Thu, Jul 2, 2009 at 5:13 PM, Schaffer, Mark E<[email protected]> wrote:
> Steve,
>
>> -----Original Message-----
>> From: Steven Archambault [mailto:[email protected]]
>> Sent: 03 July 2009 00:01
>> To: Schaffer, Mark E
>> Subject: Re: st: RE: Hausman test for clustered random vs.
>> fixed effects (again)
>>
>> Wait a second, I thought with a Chi sq test we reject the
>> null that the FE and RE coefficients are different when the
>> critical value is such that the p-value is greater or equal
>> to .05. This would give us a 5% (or more) significance that
>> the null is rejected. We get this with a lower chi-sq value.
>> It was with this logic that I am saying RE is the preferred model.
>
> There's nothing sacred about the 5% level. Some people, when constructing tables for their papers, put *s next to coefficients that are significant at the 10% level ... which happens to be your p-value.
>
> The bigger the contrasts, the smaller the p-value, and 10% implies contrasts that are large enough to make me nervous. Of course, de gustibus non est disputandum.
>
> If you want to take this further, you might consider focusing on the coefficients of interest, whatever they are. You may well find that the joint contrast between the RE and FE coefficients of interest is significant at a still smaller p-value (suggesting you dump RE), or is not at all significant (suggesting RE is preferred on efficiency grounds).
>
> -xtoverid- doesn't support tests of subsets of coefficients (I should consider adding this feature, I guess) but you can do the test by hand. It's described in the Arellano paper in the help file, and I think Vince Wiggins had a post on Statalist some time ago that describes how to do it.
>
> Cheers,
> Mark
>
>>
>> -Steve
>>
>>
>>
>> On Thu, Jul 2, 2009 at 4:47 PM, Schaffer, Mark
>> E<[email protected]> wrote:
>> > Steve,
>> >
>> >> -----Original Message-----
>> >> From: Steven Archambault [mailto:[email protected]]
>> >> Sent: 02 July 2009 22:41
>> >> To: [email protected]; Schaffer, Mark E
>> >> Cc: [email protected]; [email protected]
>> >> Subject: Re: st: RE: Hausman test for clustered random vs.
>> >> fixed effects (again)
>> >>
>> >> Mark,
>> >>
>> >> I should have commented on this earlier, but when I eye the
>> >> coefficients for both the FE and RE results, I see that
>> some of them
>> >> are quite different from one another. However, the xtoverid result
>> >> suggests RE is the one to use. Does anybody see this as a problem?
>> >> The numerator of the Hausman wald test is the difference in
>> >> coefficients of the two models. Is this not missed in the xtoverid
>> >> approach?
>> >
>> > A few things here:
>> >
>> > - The "xtoverid approach" in this case is **identical** to
>> the traditional Hausman test in concept. They are both
>> vector-of-contrast tests, the contrast being between the 9 FE
>> and RE coefficients. The **only** difference in this case
>> between the GMM stat reported by -xtoverid- and the
>> traditional Hausman stat is that the former is
>> cluster-robust. In addition to the references on this point
>> that I cited in my previous posting, you should also check
>> out Ruud's textbook, "An Introduction to Classical
>> Econometric Theory".
>> >
>> > - The test has 9 degrees of freedom because 9 coefficients
>> are being contrasted jointly. This means that some can
>> indeed be quite different, but if the others are very similar
>> then a test of the joint contrasts can be statistically insignificant.
>> >
>> > - The p-value reported by -xtoverid- is 10%, which a little
>> worrisome. If you were to do a vector-of-contrast tests
>> focusing on a subset of coefficients instead of all 9 (not
>> supported by -xtoverid- but do-able by hand), you could well
>> find that you reject the null at 5% or 1% or whatever. I
>> don't think it's straightforward to conclude that RE is the
>> estimator of choice.
>> >
>> > Hope this helps.
>> >
>> > Cheers,
>> > Mark
>> >
>> >>
>> >> I am posting my regression results to show what I am talking about
>> >> more clearly.
>> >>
>> >> Thanks for your input.
>> >> -Steve
>> >>
>> >>
>> >> Fixed-effects (within) regression Number of obs
>> >> = 404
>> >> Group variable: id_code_id Number of
>> groups =
>> >> 88
>> >>
>> >> R-sq: within = 0.2304 Obs per
>> >> group: min = 1
>> >> between = 0.4730
>> >> avg = 4.6
>> >> overall = 0.4487
>> >> max = 7
>> >>
>> >> F(9,87)
>> >> = 2.47
>> >> corr(u_i, Xb) = -0.9558 Prob > F
>> >> = 0.0148
>> >>
>> >> (Std. Err. adjusted for 88 clusters in
>> >> id_code_id)
>> >> --------------------------------------------------------------
>> >> ----------------
>> >> | Robust
>> >> lnfd | Coef. Std. Err. t P>|t|
>> [95% Conf.
>> >> Interval]
>> >> -------------+------------------------------------------------
>> >> ----------
>> >> -------------+------
>> >> lags | -.0267991 .0185982 -1.44 0.153 -.063765
>> >> .0101668
>> >> lagk | .0964571 .0353269 2.73 0.008
>> >> .0262411 .166673
>> >> lagp | .2210296 .1206562 1.83 0.070
>> >> -.0187875 .4608468
>> >> lagdr | -.0000267 .0000251 -1.06 0.291 -.0000767
>> >> .0000232
>> >> laglurb | .3483909 .1234674 2.82 0.006 .102986
>> >> .5937957
>> >> lagtra | .1109513 .1267749 0.88 0.384
>> >> -.1410275 .3629301
>> >> lagte | .0067764 .004166 1.63 0.107
>> >> -.0015039 .0150567
>> >> lagcr | .0950221 .0683074 1.39 0.168
>> >> -.0407463 .2307905
>> >> lagp | .0343752 .1291378 0.27 0.791
>> >> -.2223001 .2910506
>> >> _cons | 4.316618 1.996618 2.16 0.033
>> >> .348124 8.285112
>> >> -------------+------------------------------------------------
>> >> ----------
>> >> -------------+------
>> >> sigma_u | .44721909
>> >> sigma_e | .0595116
>> >> rho | .98260039 (fraction of variance due to u_i)
>> >> --------------------------------------------------------------
>> >> ----------------
>> >>
>> >>
>> >>
>> >> Random-effects GLS regression Number of obs
>> >> = 404
>> >> Group variable: id_code_id Number of
>> groups =
>> >> 88
>> >>
>> >> R-sq: within = 0.1792 Obs per
>> >> group: min = 1
>> >> between = 0.5074
>> >> avg = 4.6
>> >> overall = 0.5017
>> >> max = 7
>> >>
>> >> Random effects u_i ~ Gaussian Wald chi2(9)
>> >> = 48.97
>> >> corr(u_i, X) = 0 (assumed) Prob > chi2
>> >> = 0.0000
>> >>
>> >> (Std. Err. adjusted for clustering on
>> >> id_code_id)
>> >> --------------------------------------------------------------
>> >> ----------------
>> >> | Robust
>> >> lnfd | Coef. Std. Err. z P>|z|
>> [95% Conf.
>> >> Interval]
>> >> -------------+------------------------------------------------
>> >> ----------
>> >> -------------+------
>> >> lags | -.01138 .0135958 -0.84 0.403 -.0380274
>> >> .0152673
>> >> lagk | .0115314 .0180641 0.64 0.523
>> >> -.0238735 .0469363
>> >> lagp | .2551701 .119322 2.14 0.032
>> >> .0213033 .4890369
>> >> lagdr | -6.17e-06 .0000153 -0.40 0.686 -.0000361
>> >> .0000238
>> >> laglurb | .0657802 .0153923 4.27 0.000 .0356119
>> >> .0959486
>> >> lagtra | .0022183 .0579203 0.04 0.969
>> >> -.1113034 .11574
>> >> lagte | .0048012 .0016128 2.98 0.003
>> >> .00164 .0079623
>> >> lagcr | .1051833 .045994 2.29 0.022
>> >> .0150368 .1953298
>> >> lagp | .184373 .1191063 1.55 0.122
>> >> -.0490711 .4178171
>> >> _cons | 9.071133 .2322309 39.06 0.000
>> >> 8.615968 9.526297
>> >> -------------+------------------------------------------------
>> >> ----------
>> >> -------------+------
>> >> sigma_u | .10617991
>> >> sigma_e | .0595116
>> >> rho | .76095591 (fraction of variance due to u_i)
>> >> --------------------------------------------------------------
>> >> ----------------
>> >>
>> >> . xtoverid;
>> >>
>> >> Test of overidentifying restrictions: fixed vs random effects
>> >> Cross-section time-series model: xtreg re robust Sargan-Hansen
>> >> statistic 14.684 Chi-sq(9) P-value = 0.1000
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Sat, Jun 27, 2009 at 11:31 AM, Schaffer, Mark
>> >> E<[email protected]> wrote:
>> >> > Steve,
>> >> >
>> >> >> -----Original Message-----
>> >> >> From: [email protected]
>> >> >> [mailto:[email protected]] On Behalf
>> Of Steven
>> >> >> Archambault
>> >> >> Sent: 27 June 2009 00:26
>> >> >> To: [email protected]; [email protected];
>> >> >> [email protected]
>> >> >> Subject: st: Hausman test for clustered random vs. fixed effects
>> >> >> (again)
>> >> >>
>> >> >> Hi all,
>> >> >>
>> >> >> I know this has been discussed before, but in STATA 10
>> >> (and versions
>> >> >> before 9 I understand) the canned procedure for Hausman
>> test when
>> >> >> comparing FE and RE models cannot be run when the data
>> >> analysis uses
>> >> >> clustering (and by default corrects for robust errors
>> in STATA 10).
>> >> >> This is the error received
>> >> >>
>> >> >> "hausman cannot be used with vce(robust), vce(cluster cvar), or
>> >> >> p-weighted data"
>> >> >>
>> >> >> My question is whether or not the approach of using xtoverid to
>> >> >> compare FE and RE models (analyzed using the clustered and
>> >> by default
>> >> >> robust approach in STATA 10) is accepted in the
>> literature. This
>> >> >> approach produces the Sargan-Hansen stat, which is
>> typically used
>> >> >> with analyses that have instrumentalized variables and need an
>> >> >> overidentification test. For the sake of publishing I am
>> >> wondering if
>> >> >> it is better just not to worry about heteroskedaticity,
>> and avoid
>> >> >> clustering in the first place (even though
>> >> heteroskedaticity likely
>> >> >> exists)? Or, alternatively one could just calculate the
>> >> Hausman test
>> >> >> by hand following the clustered analyses.
>> >> >>
>> >> >> Thanks for your insight.
>> >> >
>> >> > It's very much accepted in the literature. In the
>> -xtoverid- help
>> >> > file, see especially the paper by Arellano and the book
>> by Hayashi.
>> >> >
>> >> > If you suspect heteroskedasticity or clustered errors,
>> >> there really is
>> >> > no good reason to go with a test (classic Hausman) that is
>> >> invalid in
>> >> > the presence of these problems. The GMM -xtoverid-
>> approach is a
>> >> > generalization of the Hausman test, in the following sense:
>> >> >
>> >> > - The Hausman and GMM tests of fixed vs. random effects
>> >> have the same
>> >> > degrees of freedom. This means the result cited by Hayashi
>> >> (and due
>> >> > to Newey, if I recall) kicks in, namely...
>> >> >
>> >> > - Under the assumption of homoskedasticity and independent
>> >> errors, the
>> >> > Hausman and GMM test statistics are numerically identical.
>> >> Same test.
>> >> >
>> >> > - When you loosen the iid assumption and allow
>> >> heteroskedasticity or
>> >> > dependent data, the robust GMM test is the natural
>> generalization.
>> >> >
>> >> > Hope this helps.
>> >> >
>> >> > Cheers,
>> >> > Mark (author of -xtoverid-)
>> >> >
>> >> >> *
>> >> >> * For searches and help try:
>> >> >> * http://www.stata.com/help.cgi?search
>> >> >> * http://www.stata.com/support/statalist/faq
>> >> >> * http://www.ats.ucla.edu/stat/stata/
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Heriot-Watt University is a Scottish charity registered
>> >> under charity
>> >> > number SC000278.
>> >> >
>> >> >
>> >> > *
>> >> > * For searches and help try:
>> >> > * http://www.stata.com/help.cgi?search
>> >> > * http://www.stata.com/support/statalist/faq
>> >> > * http://www.ats.ucla.edu/stat/stata/
>> >> >
>> >>
>> >
>> >
>> > --
>> > Heriot-Watt University is a Scottish charity registered
>> under charity
>> > number SC000278.
>> >
>> >
>>
>
>
> --
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
>
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
