|
|  |
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
RE: st: ivreset
Hi all. I think I spoke too soon. The differences between the
different approaches to rescaling are driven, I think, by numerical
precision problems and collinearities.
Here are the 3 versions of the basic ovtest using squares and cubes: no
rescaling, rescaling yhat and then squaring and cubing, and squaring and
cubing followed by rescaling. This uses the code that I posted to the
list earlier today. All three generate the same results. When a yhat^4
is included, as in the example I posted earlier, is when collinearities
arise and different test statistics can be generated.
--Mark
. * Squared and cubed
. * Unrescaled RESET
. qui regress lw s yhat2 yhat3
. testparm yhat*
( 1) yhat2 = 0
( 2) yhat3 = 0
F( 2, 754) = 0.87
Prob > F = 0.4188
. * yhat that is first ^2, ^3, then rescaled
. * Output identical to unrescaled RESET
. qui regress lw s yhatrr2 yhatrr3
. testparm yhat*
( 1) yhatrr2 = 0
( 2) yhatrr3 = 0
F( 2, 754) = 0.87
Prob > F = 0.4188
. * yhat that is first rescaled, then ^2, ^3
. * Output different from unrescaled RESET
. qui regress lw s yhatr2 yhatr3
. testparm yhat*
( 1) yhatr2 = 0
( 2) yhatr3 = 0
F( 2, 754) = 0.87
Prob > F = 0.4188
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> Schaffer, Mark E
> Sent: 20 March 2006 16:57
> To: [email protected]
> Subject: RE: st: ivreset
>
> Arne,
>
> I think you are onto something here. The rescaling should
> perhaps take place *after* creating the polynomials, not before.
>
> That said, Stata's official -ovtest- also rescales first,
> then creates the polynomials. When I wrote -ivreset-, I had
> used replication of the output of -ovtest- as a check, and I
> used the same approach to rescaling yhat, namely rescale and
> then create polynomials (rather than create polynomials and
> then rescale).
>
> Here is an example that demonstrates that (a) rescaling after
> creating the polynomials leaves the reset statistic
> unchanged, and (b) rescaling before creating the polynomials
> replicates the output of official -ovtest-. Note that there
> are collinearity problems with (a) even after rescaling,
> which makes me hesitate...
>
> Does anyone else want to comment?
>
> --Mark
>
> *********** do code ***************
>
> use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta,
> clear capture drop yhat* qui regress lw s qui predict double yhat
> * yhatr=rescaled yhat
> sum yhat, meanonly
> qui gen double yhatr = (yhat-r(min))/(r(max)-r(min)) qui gen
> double yhat2=yhat^2 qui gen double yhat3=yhat^3 qui gen
> double yhat4=yhat^4 qui gen double yhatr2=yhatr^2 qui gen
> double yhatr3=yhatr^3 qui gen double yhatr4=yhatr^4
> * yhatrr2=rescaled yhat2
> * yhatrr3=rescaled yhat3
> * yhatrr4=rescaled yhat4
> sum yhat2, meanonly
> qui gen double yhatrr2 = (yhat2-r(min))/(r(max)-r(min)) sum
> yhat3, meanonly qui gen double yhatrr3 =
> (yhat3-r(min))/(r(max)-r(min)) sum yhat4, meanonly qui gen
> double yhatrr4 = (yhat4-r(min))/(r(max)-r(min)) sum yhat*
>
> * Unrescaled RESET
> qui regress lw s yhat2 yhat3 yhat4
> testparm yhat*
> * yhat that is first ^2, ^3, ^4, then rescaled
> * Output identical to unrescaled RESET
> qui regress lw s yhatrr2 yhatrr3 yhatrr4 testparm yhat*
> * yhat that is first rescaled, then ^2, ^3, ^4
> * Output different from unrescaled RESET qui regress lw s
> yhatr2 yhatr3 yhatr4 testparm yhat*
> * Stata's built-in ovtest
> * Output again different from unrescaled RESET qui regress lw s Ovtest
>
> ************* output ****************
>
>
> . use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta,
> clear (Wages of Very Young Men, Zvi Griliches, J.Pol.Ec. 1976)
>
> . capture drop yhat*
>
> . qui regress lw s
>
> . qui predict double yhat
>
> . * yhatr=rescaled yhat
> . sum yhat, meanonly
>
> . qui gen double yhatr = (yhat-r(min))/(r(max)-r(min))
>
> . qui gen double yhat2=yhat^2
>
> . qui gen double yhat3=yhat^3
>
> . qui gen double yhat4=yhat^4
>
> . qui gen double yhatr2=yhatr^2
>
> . qui gen double yhatr3=yhatr^3
>
> . qui gen double yhatr4=yhatr^4
>
> . * yhatrr2=rescaled yhat2
> . * yhatrr3=rescaled yhat3
> . * yhatrr4=rescaled yhat4
> . sum yhat2, meanonly
>
> . qui gen double yhatrr2 = (yhat2-r(min))/(r(max)-r(min))
>
> . sum yhat3, meanonly
>
> . qui gen double yhatrr3 = (yhat3-r(min))/(r(max)-r(min))
>
> . sum yhat4, meanonly
>
> . qui gen double yhatrr4 = (yhat4-r(min))/(r(max)-r(min))
>
> . sum yhat*
>
> Variable | Obs Mean Std. Dev. Min Max
> -------------+--------------------------------------------------------
> yhat | 758 5.686739 .2156493 5.261107 6.130727
> yhatr | 758 .4894459 .2479809 0 1
> yhat2 | 758 32.38544 2.473995 27.67924 37.58581
> yhat3 | 758 184.7003 21.3139 145.6234 230.4284
> yhat4 | 758 1054.929 163.4247 766.1405 1412.693
> -------------+--------------------------------------------------------
> yhatr2 | 758 .3009707 .2769761 0 1
> yhatr3 | 758 .2142687 .2681644 0 1
> yhatr4 | 758 .1671979 .2530434 0 1
> yhatrr2 | 758 .4750582 .2497327 0 1
> yhatrr3 | 758 .4607848 .2513286 0 1
> -------------+--------------------------------------------------------
> yhatrr4 | 758 .4466593 .252763 0 1
>
> .
> . * Unrescaled RESET
> . qui regress lw s yhat2 yhat3 yhat4
>
> . testparm yhat*
>
> ( 1) yhat2 = 0
> ( 2) yhat3 = 0
> ( 3) yhat4 = 0
> Constraint 1 dropped
>
> F( 2, 754) = 0.87
> Prob > F = 0.4191
>
> . * yhat that is first ^2, ^3, ^4, then rescaled . * Output
> identical to unrescaled RESET . qui regress lw s yhatrr2
> yhatrr3 yhatrr4
>
> . testparm yhat*
>
> ( 1) yhatrr2 = 0
> ( 2) yhatrr3 = 0
> ( 3) yhatrr4 = 0
> Constraint 1 dropped
>
> F( 2, 754) = 0.87
> Prob > F = 0.4191
>
> . * yhat that is first rescaled, then ^2, ^3, ^4 . * Output
> different from unrescaled RESET . qui regress lw s yhatr2
> yhatr3 yhatr4
>
> . testparm yhat*
>
> ( 1) yhatr2 = 0
> ( 2) yhatr3 = 0
> ( 3) yhatr4 = 0
>
> F( 3, 753) = 0.59
> Prob > F = 0.6216
>
> . * Stata's built-in ovtest
> . * Output again different from unrescaled RESET . qui regress lw s
>
> . ovtest
>
> Ramsey RESET test using powers of the fitted values of lw
> Ho: model has no omitted variables
> F(3, 753) = 0.59
> Prob > F = 0.6216
>
> .
> end of do-file
>
> *************************************
>
> > -----Original Message-----
> > From: [email protected]
> > [mailto:[email protected]] On Behalf Of
> Arne Risa
> > Hole
> > Sent: 20 March 2006 16:03
> > To: [email protected]
> > Subject: Re: st: ivreset
> >
> > Hi Mark,
> >
> > Thank you very much for the clarifications. The rescaling
> does have an
> > impact on the test statistic, however; this can be seen from using
> > Austin's code and comparing the results with and without the line:
> >
> > replace yhat = (yhat-r(min))/(r(max)-r(min))
> >
> > So even when the correct "optimal" forecast of yhat is used
> > (yhat=X-hat*beta-hat), rescaling the forecast affects the
> result. This
> > is not a problem in the case of -ovtest-, however, since the Reset
> > test statistic is invariant to the rescaling in the OLS case.
> >
> > Sorry for going on about this, but it seems to me that
> since the two
> > statistics differ the correct statistic is the one without the
> > rescaling of the (yhat=X-hat*beta-hat) forecast (even
> though this may
> > introduce numerical precision problems in some cases).
> >
> > Cheers,
> > Arne
> >
> > On 20/03/06, Schaffer, Mark E <[email protected]> wrote:
> > > Arne,
> > >
> > > > -----Original Message-----
> > > > From: [email protected]
> > > > [mailto:[email protected]] On Behalf
> > Of Arne Risa
> > > > Hole
> > > > Sent: 18 March 2006 11:43
> > > > To: [email protected]
> > > > Subject: RE: st: ivreset
> > > >
> > > > Austin, Mark,
> > > >
> > > > Thank you both for your replies, rescaling the forecast did the
> > > > trick (sorry for the bad formatting of my code before,
> it looked
> > > > fine in my email programme).
> > > >
> > > > I understand the motivation behind the rescaling, but I'm
> > slightly
> > > > concerned about the fact that it produces a different
> > test statistic
> > > > compared to using the actual forecast. Note that this
> is not the
> > > > case when using the Reset test following OLS - the test
> > statistic is
> > > > invariant to the rescaling in this case.
> > > >
> > > > I would think that since the two approaches (rescaling/ no
> > > > rescaling) produce different results, the correct test
> > statistic is
> > > > that using the actual forecast?
> > >
> > > There are two different issues here. Rescaling is one, and
> > it is, in
> > > some sense, a side issue. The problem is that sometimes
> > the yhat has
> > > large-ish values, and the higher order polynomials of
> yhat that are
> > > included in the artificial regression can get so big that the
> > > regression fails for numerical precision reasons. Stata's
> > own version
> > > of the reset test, -ovtest-, also does this rescaling. The test
> > > statistic is, of course, invariant in theory to the units
> > used and hence to rescaling.
> > >
> > > The other issue is the one you might have missed. As
> > Austin pointed
> > > out, the IV version of the RESET test cannot use standard fitted
> > > values that would be generated by -predict- after estimation using
> > > -ivreg- or -ivreg2-. These are yhat=X*beta-hat, and the
> problem is
> > > that X includes some endogenous regressors.
> > >
> > > As the help file for -ivreset- explains, there are two
> alternatives.
> > > One is to use reduced form predictions for yhat, i.e.,
> regress y on
> > > all the exogenous variables (including the excluded
> > instruments) and
> > > then use -predict-. The other is to get what Pesaran and
> > Taylor call
> > > the "optimal forecast" of yhat. This is not yhat=X*beta-hat, but
> > > yhat=X-hat*beta-hat, where X-hat includes the reduced form
> > predicted
> > > values of the endogenous regressors (rather than the
> actual values).
> > > The code that Austin kindly posted to Statalast implemented
> > the latter.
> > >
> > > Cheers,
> > > Mark
> > >
> > > > Cheers
> > > > Arne
> > > >
> > > > On Mar 17 2006, Schaffer, Mark E wrote:
> > > >
> > > > > Austin, Arne,
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Austin Nichols [mailto:[email protected]]
> > > > > > Sent: 16 March 2006 23:30
> > > > > > To: [email protected]
> > > > > > Subject: Re: st: ivreset
> > > > > >
> > > > > > -findit ivreset- then -help ivreset- when installed has a
> > > > excellent
> > > > > > exposition that begins:
> > > > > >
> > > > > > As Pagan and Hall (1983) and Pesaran and Taylor (1999)
> > > > point out, a
> > > > > > RESET test for an IV regression cannot use the standard
> > > > IV predicted
> > > > > > values X*beta-hat, because X includes endogenous
> > > > regressors that are
> > > > > > correlated with u.
> > > > > >
> > > > > > Try this code instead:
> > > > > >
> > > > > > use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta
> > > > > > qui ivreg2 lw s expr tenure rns smsa (iq=med kww) predict
> > > > ytilde mat
> > > > > > b=e(b) mat li b qui regress iq s expr tenure rns
> smsa med kww
> > > > > > qui predict double xh gen yhat=ytil-b[1,1]*iq+b[1,1]*xh sum
> > > > > > yhat, meanonly qui replace yhat =
> > (yhat-r(min))/(r(max)-r(min))
> > > > > > qui gen double yhat2=yhat^2 qui ivreg2 lw s expr
> > tenure rns smsa
> > > > > > yhat2 (iq=med kww) test yhat2 qui ivreg2 lw s expr
> tenure rns
> > > > smsa (iq=med
> > > > > > kww) ivreset
> > > > > >
> > > > > > Now, as to why
> > > > > > replace yhat = (yhat-r(min))/(r(max)-r(min)) I cannot
> > > > tell you, but
> > > > > > it's in ivreset.ado
> > > > >
> > > > > It's to rescale yhat so that when it's squared,
> cubed, etc., it
> > > > > doesn't get wildly out of scale relative to the other
> > regressors.
> > > > > This can cause problems for the regression that includes
> > > > these terms.
> > > > >
> > > > > Cheers,
> > > > > Mark
> > > > >
> > > > >
> > > > > > On 16 Mar 2006 19:10:13 +0000, Arne Risa Hole wrote:
> > > > > > > I am using ivreset to do a Pesaran-Taylor Reset test
> > > > after ivreg2.
> > > > > > > However, I am not able to replicate the result
> from ivreset
> > > > > > manually. For example:
> > > > > > >
> > > > > > > use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta
> > > > > >
> > > > > >
> > > > >
> > > > > *
> > > > > * For searches and help try:
> > > > > * http://www.stata.com/support/faqs/res/findit.html
> > > > > * http://www.stata.com/support/statalist/faq
> > > > > * http://www.ats.ucla.edu/stat/stata/
> > > > >
> > > >
> > > > *
> > > > * For searches and help try:
> > > > * http://www.stata.com/support/faqs/res/findit.html
> > > > * http://www.stata.com/support/statalist/faq
> > > > * http://www.ats.ucla.edu/stat/stata/
> > > >
> > > >
> > >
> > > *
> > > * For searches and help try:
> > > * http://www.stata.com/support/faqs/res/findit.html
> > > * http://www.stata.com/support/statalist/faq
> > > * http://www.ats.ucla.edu/stat/stata/
> > >
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/support/faqs/res/findit.html
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
> >
>
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
>
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
