st: Re: About weakiv

[Chau Tak Wai <chau.takwai@gmail.com>]

Mark, 

Thank you very much for your reply. Have you checked that the performance of AR in weakiv generally agrees or outperforms 
the case if we directly implement this with an F or Wald test with robust standard errors for the regression of y-b0*x on z? 
In my simulation, it looks like it over-rejects more, but maybe it's not the general case.

Thanks for your attention and reply!

Tak Wai

=============================
Tak Wai,

I don't think there's any bug in how -weakiv- calculates the AR stat with a small-sample adjustment in the robust case - see below.  What you're picking up is probably just how that test performs in your particular MC setup.

The built-in small-sample adjustment in -weakiv- is very simple: the chi-sq stat uses (N-L) instead of N, where L = #excluded instruments + #exogenous regressors + 1 for a constant (if included).

Below is an illustration using the toy auto dataset.  I use -ivreg2- to do the AR regression because it's an easy way of getting the large sample chi-sq test stat and then doing the weakiv-style small-sample adjustment.

It would be straightforward to add this to your code and confirm.  My guess is that you are finding that in your MC design, the AR test using Eicker-Huber-White-heteroskedastic-robust SEs and the small-sample adjustment described above overrejects.  That is, this is a general finding and not code-specific.

--Mark

**** code ****
sysuse auto
qui weakiv ivreg2 price trunk (mpg = weight turn), rob small
di e(ar_chi2)
qui ivreg2 price trunk weight turn, rob
qui test weight turn
di r(chi2) * (e(N)-e(df_m)-e(cons))/e(N)

**** output ****
. sysuse auto
(1978 Automobile Data)

. qui weakiv ivreg2 price trunk (mpg = weight turn), rob small

. di e(ar_chi2)
17.243124

. qui ivreg2 price trunk weight turn, rob

. qui test weight turn

. di r(chi2) * (e(N)-e(df_m)-e(cons))/e(N)
17.243124


> Hi, all,
>
> I have updated the program to evaluate the test in a few ways. 1. weakiv
> with robust; 2. weakiv without robust; 3. condivreg (no robust); 4. F
> stat in AR form with robust. The codes and results are shown below. I
> have increased the sample size to 300, and the AR statistic provided by
> weakiv with robust seem to over-reject the null relative to other
> methods. This may also lead to over-rejection for the CLR test. If there
> seems to be a bug in the robust version, hope the following results may
> help debugging it.
>
> capture program drop sim_iv_clr_15
> program define sim_iv_clr_15, rclass
> version 11
> syntax [, obs(integer 100) mu2k(real 3) b(real 1) rho(real 0.5) ]
> drop _all
> set obs `obs'
> tempvar y x z1 z2 z3 z4 z5 z6 z7 z8 z9 z10 z11 z12 z13 z14 z15 u yt
> gen `z1' = rnormal(0,1)
> gen `z2' = rnormal(0,1)
> gen `z3' = rnormal(0,1)
> gen `z4' = rnormal(0,1)
> gen `z5' = rnormal(0,1)
> gen `z6' = rnormal(0,1)
> gen `z7' = rnormal(0,1)
> gen `z8' = rnormal(0,1)
> gen `z9' = rnormal(0,1)
> gen `z10' = rnormal(0,1)
> gen `z11' = rnormal(0,1)
> gen `z12' = rnormal(0,1)
> gen `z13' = rnormal(0,1)
> gen `z14' = rnormal(0,1)
> gen `z15' = rnormal(0,1)
>
> gen `u' = rnormal(0,1)
>
> gen `x' =
> sqrt(`mu2k'/`obs')*(`z1'+`z2'+`z3'+`z4'+`z5'+`z6'+`z7'+`z8'+`z9'+`z10'+`z11'+`z12'+`z13'+`z14'+`z15')+`u'
> gen `y' = `b'*`x'+(`rho')*`u'+sqrt(1-`rho'^2)*rnormal(0,1)
>
> local z `z1' `z2' `z3' `z4' `z5' `z6' `z7' `z8' `z9' `z10' `z11' `z12'
> `z13' `z14' `z15'
> *check first stage
> regress `x' `z'
> return scalar fsf=e(F)
>
> ivregress 2sls `y' (`x' = `z'), robust
> return scalar beta2sls=_b[`x']
> return scalar se2sls=_se[`x']
>
> ivregress liml `y' (`x' = `z'), robust
> return scalar betaliml=_b[`x']
> return scalar seliml=_se[`x']
>
> ivreg2 `y' (`x'=`z' ), fuller(1) robust
> return scalar betaf1=_b[`x']
> return scalar sef1=_se[`x']
>
> ** check the related tests at null of true value
> ivregress 2sls `y' (`x' = `z'), robust
> weakiv, null(1.0) small
> return scalar clrp=e(clr_p)
> return scalar arp=e(ar_p)
> return scalar wp=e(wald_p)
>
> **cross check: without using robust
> ivregress 2sls `y' (`x' = `z')
> weakiv, null(1.0) small
> return scalar clrp2=e(clr_p)
> return scalar arp2=e(ar_p)
> return scalar wp2=e(wald_p)
>
> ** use another ado routine, not robust
> condivreg `y' (`x'=`z' ), ar test(1.0)
> return scalar clrp3=e(p_LR)
> return scalar arp3=e(p_AR)
>
> ** test AR using traditional method, robust var
> gen `yt'=`y'-`b'*`x'
> regress `yt' `z', robust
> return scalar arF4=e(F)
>
> ivregress 2sls `y' (`x' = `z'), robust
> weakiv, null(0.75) small
> return scalar clrpa=e(clr_p)
> return scalar arpa=e(ar_p)
> return scalar wpa=e(wald_p)
>
> ivregress 2sls `y' (`x' = `z'), robust
> weakiv, null(1.25) small
> return scalar clrpb=e(clr_p)
> return scalar arpb=e(ar_p)
> return scalar wpb=e(wald_p)
>
> end
>
>
> local betatrue 1.0
> local nobs 300
> simulate fsf=r(fsf) beta2sls=r(beta2sls) se2sls=r(se2sls) /*
> */ betaliml=r(betaliml) seliml=r(seliml) betaf1=r(betaf1) sef1=r(sef1)/*
> */ clrp=r(clrp) arp=r(arp) wp=r(wp) clrp2=r(clrp2) arp2=r(arp2)
> wp2=r(wp2) clrp3=r(clrp3) arp3=r(arp3) arF4=r(arF4)/*
> */ clrpa=r(clrpa) arpa=r(arpa) wpa=r(wpa) clrpb=r(clrpb) arpb=r(arpb)
> wpb=r(wpb), /*
> */ reps(1000): sim_iv_clr_15, obs(`nobs') b(`betatrue') mu2k(1.0) rho(0.1)
>
> **construct test statistics and results
> gen t2sls=(beta2sls-`betatrue')/se2sls
> gen tliml=(betaliml-`betatrue')/seliml
> gen tf1=(betaf1-`betatrue')/sef1
>
> gen t2slsa=(beta2sls-`betatrue'+0.25)/se2sls
> gen tlimla=(betaliml-`betatrue'+0.25)/seliml
> gen tf1a=(betaf1-`betatrue'+0.25)/sef1
> gen t2slsb=(beta2sls-`betatrue'-0.25)/se2sls
> gen tlimlb=(betaliml-`betatrue'-0.25)/seliml
> gen tf1b=(betaf1-`betatrue'-0.25)/sef1
>
> gen ttest2sls=(abs(t2sls)>1.96)
> gen ttestliml=(abs(tliml)>1.96)
> gen ttestf1=(abs(tf1)>1.96)
> gen clrtest=(clrp<0.05)
> gen artest=(arp<0.05)
> gen wtest=(wp<0.05)
> gen clrtest2=(clrp2<0.05)
> gen artest2=(arp2<0.05)
> gen wtest2=(wp2<0.05)
> gen clrtest3=(clrp3<0.05)
> gen artest3=(arp3<0.05)
> gen artest4=(arF4>invFtail(15,`nobs'-16,0.05))
>
> gen ttest2slsa=(abs(t2slsa)>1.96)
> gen ttestlimla=(abs(tlimla)>1.96)
> gen ttestf1a=(abs(tf1a)>1.96)
> gen clrtesta=(clrpa<0.05)
> gen artesta=(arpa<0.05)
> gen wtesta=(wpa<0.05)
>
> gen ttest2slsb=(abs(t2slsb)>1.96)
> gen ttestlimlb=(abs(tlimlb)>1.96)
> gen ttestf1b=(abs(tf1b)>1.96)
> gen clrtestb=(clrpb<0.05)
> gen artestb=(arpb<0.05)
> gen wtestb=(wpb<0.05)
>
> su
>
> Variable | Obs Mean Std. Dev. Min Max
> -------------+--------------------------------------------------------
> fsf | 1000 1.989075 .6686453 .5584267 4.559362
> beta2sls | 1000 1.046762 .1902355 .400066 1.668987
> se2sls | 1000 .1928713 .0385838 .1219964 .4041867
> betaliml | 1000 .9003882 2.734562 -78.64992 12.28343
> seliml | 1000 211.6659 6518.227 .1293966 206121.8
> -------------+--------------------------------------------------------
> betaf1 | 1000 1.002532 .4538258 -.7728926 3.102209
> sef1 | 1000 .5586889 .7411543 .1270225 5.205231
> clrp | 1000 .4279222 .2941029 .0000795 .9686568
> arp | 1000 .3841276 .2947135 2.71e-06 .9976762
> wp | 1000 .4900556 .2838048 .0016207 .9998908
> -------------+--------------------------------------------------------
> clrp2 | 1000 .4726739 .2929098 .0007298 .9756876
> arp2 | 1000 .4613457 .293643 .0001723 .9985789
> wp2 | 1000 .4919608 .2827918 .0009518 .9998832
> clrp3 | 1000 .4931547 .2886754 .0013392 .9787179
> arp3 | 1000 .5050222 .2926132 .0004048 .9989961
> -------------+--------------------------------------------------------
> arF4 | 1000 1.126271 .4503785 .2536111 3.402767
> clrpa | 1000 .3805249 .3046617 1.62e-06 .9678711
> arpa | 1000 .3481354 .290572 8.98e-07 .9931965
> wpa | 1000 .2324543 .2744134 1.10e-06 .9984452
> clrpb | 1000 .371495 .3055804 2.81e-06 .959181
> -------------+--------------------------------------------------------
> arpb | 1000 .3407169 .2877087 1.78e-06 .9990297
> wpb | 1000 .348814 .3055344 .0000736 .9968536
> t2sls | 1000 .2679759 .9578728 -2.616042 3.152151
> tliml | 1000 .0361009 .8693161 -2.454555 2.892277
> tf1 | 1000 .0512138 .8852093 -2.484029 2.921904
> -------------+--------------------------------------------------------
> t2slsa | 1000 1.611938 1.021119 -1.267857 4.872668
> tlimla | 1000 .7455774 .9830085 -1.574504 4.198492
> tf1a | 1000 .8109588 .9943123 -1.550451 4.269866
> t2slsb | 1000 -1.075986 .9566182 -3.964227 1.77621
> tlimlb | 1000 -.6733755 .9044783 -3.64242 1.739805
> -------------+--------------------------------------------------------
> tf1b | 1000 -.7085311 .9179014 -3.680793 1.75169
> ttest2sls | 1000 .043 .2029586 0 1
> ttestliml | 1000 .016 .1255379 0 1
> ttestf1 | 1000 .018 .1330176 0 1
> clrtest | 1000 .11 .3130463 0 1
> -------------+--------------------------------------------------------
> artest | 1000 .144 .3512654 0 1
> wtest | 1000 .043 .2029586 0 1
> clrtest2 | 1000 .063 .2430845 0 1
> artest2 | 1000 .077 .2667248 0 1
> wtest2 | 1000 .042 .2006895 0 1
> -------------+--------------------------------------------------------
> clrtest3 | 1000 .051 .2201078 0 1
> artest3 | 1000 .062 .2412762 0 1
> artest4 | 1000 .11 .3130463 0 1
> ttest2slsa | 1000 .385 .4868388 0 1
> ttestlimla | 1000 .13 .3364717 0 1
> -------------+--------------------------------------------------------
> ttestf1a | 1000 .143 .350248 0 1
> clrtesta | 1000 .186 .3893014 0 1
> artesta | 1000 .183 .38686 0 1
> wtesta | 1000 .385 .4868388 0 1
> ttest2slsb | 1000 .18 .3843797 0 1
> -------------+--------------------------------------------------------
> ttestlimlb | 1000 .074 .2619019 0 1
> ttestf1b | 1000 .086 .2805043 0 1
> clrtestb | 1000 .175 .3801572 0 1
> artestb | 1000 .192 .3940702 0 1
> wtestb | 1000 .18 .3843797 0 1
>
> ******* rho=0.5
> clear
> local betatrue 1.0
> local nobs 300
> simulate fsf=r(fsf) beta2sls=r(beta2sls) se2sls=r(se2sls) /*
> */ betaliml=r(betaliml) seliml=r(seliml) betaf1=r(betaf1) sef1=r(sef1)/*
> */ clrp=r(clrp) arp=r(arp) wp=r(wp) clrp2=r(clrp2) arp2=r(arp2)
> wp2=r(wp2) clrp3=r(clrp3) arp3=r(arp3) arF4=r(arF4)/*
> */ clrpa=r(clrpa) arpa=r(arpa) wpa=r(wpa) clrpb=r(clrpb) arpb=r(arpb)
> wpb=r(wpb), /*
> */ reps(1000): sim_iv_clr_15, obs(`nobs') b(`betatrue') mu2k(1.0) rho(0.5)
>
> . su
>
> Variable | Obs Mean Std. Dev. Min Max
> -------------+--------------------------------------------------------
> fsf | 1000 2.024847 .6701931 .5740368 4.831673
> beta2sls | 1000 1.24525 .1796135 .5240726 1.854688
> se2sls | 1000 .173738 .0368764 .0928082 .3488971
> betaliml | 1000 .0126125 28.26287 -889.5311 26.34858
> seliml | 1000 327270.1 1.03e+07 .1150639 3.26e+08
> -------------+--------------------------------------------------------
> betaf1 | 1000 1.027248 .4093937 -.7209634 3.021829
> sef1 | 1000 .5349395 .7022094 .112703 5.234337
> clrp | 1000 .4391295 .3053888 .0000104 .9532101
> arp | 1000 .3849927 .2981145 1.01e-06 .9954741
> wp | 1000 .2573638 .2911204 2.16e-07 .9941927
> -------------+--------------------------------------------------------
> clrp2 | 1000 .4766389 .2989577 .0000587 .9668206
> arp2 | 1000 .4532889 .2976167 .0002123 .9987825
> wp2 | 1000 .2582972 .2911485 7.71e-07 .9941952
> clrp3 | 1000 .4940738 .2948846 .0001081 .9697951
> arp3 | 1000 .4964008 .2963669 .0004908 .9991418
> -------------+--------------------------------------------------------
> arF4 | 1000 1.131203 .4643455 .2852084 3.564459
> clrpa | 1000 .3813428 .3045638 7.62e-07 .9599406
> arpa | 1000 .3544708 .2948629 1.44e-07 .9985713
> wpa | 1000 .0615626 .1487517 2.36e-13 .989473
> clrpb | 1000 .3573862 .3093374 5.02e-06 .9679686
> -------------+--------------------------------------------------------
> arpb | 1000 .3306423 .2857347 .0000268 .9998677
> wpb | 1000 .4925148 .2864962 .0006367 .9990597
> t2sls | 1000 1.543295 1.139573 -1.653001 5.184616
> tliml | 1000 .2409014 .9227463 -1.751008 4.002587
> tf1 | 1000 .3212997 .944274 -1.7258 4.053029
> -------------+--------------------------------------------------------
> t2slsa | 1000 3.042114 1.331998 -.7846958 7.326546
> tlimla | 1000 .9852943 1.196789 -1.067041 5.599939
> tf1a | 1000 1.123272 1.21262 -.9787121 5.766745
> t2slsb | 1000 .0444763 .9989699 -2.660775 3.415459
> tlimlb | 1000 -.5034916 .774259 -2.587835 2.566712
> -------------+--------------------------------------------------------
> tf1b | 1000 -.480673 .795856 -2.611754 2.672471
> ttest2sls | 1000 .355 .4787528 0 1
> ttestliml | 1000 .051 .2201078 0 1
> ttestf1 | 1000 .06 .2376057 0 1
> clrtest | 1000 .117 .3215811 0 1
> -------------+--------------------------------------------------------
> artest | 1000 .156 .3630369 0 1
> wtest | 1000 .355 .4787528 0 1
> clrtest2 | 1000 .069 .253581 0 1
> artest2 | 1000 .09 .286325 0 1
> wtest2 | 1000 .349 .4768925 0 1
> -------------+--------------------------------------------------------
> clrtest3 | 1000 .061 .2394501 0 1
> artest3 | 1000 .07 .2552747 0 1
> artest4 | 1000 .111 .3142893 0 1
> ttest2slsa | 1000 .776 .4171307 0 1
> ttestlimla | 1000 .21 .407512 0 1
> -------------+--------------------------------------------------------
> ttestf1a | 1000 .238 .4260722 0 1
> clrtesta | 1000 .169 .3749394 0 1
> artesta | 1000 .186 .3893014 0 1
> wtesta | 1000 .776 .4171307 0 1
> ttest2slsb | 1000 .039 .1936918 0 1
> -------------+--------------------------------------------------------
> ttestlimlb | 1000 .025 .1562031 0 1
> ttestf1b | 1000 .024 .1531256 0 1
> clrtestb | 1000 .211 .4082225 0 1
> artestb | 1000 .202 .4016931 0 1
> wtestb | 1000 .039 .1936918 0 1
>
> ****rho=0.9
> clear
> local betatrue 1.0
> local nobs 300
> simulate fsf=r(fsf) beta2sls=r(beta2sls) se2sls=r(se2sls) /*
> */ betaliml=r(betaliml) seliml=r(seliml) betaf1=r(betaf1) sef1=r(sef1)/*
> */ clrp=r(clrp) arp=r(arp) wp=r(wp) clrp2=r(clrp2) arp2=r(arp2)
> wp2=r(wp2) clrp3=r(clrp3) arp3=r(arp3) arF4=r(arF4)/*
> */ clrpa=r(clrpa) arpa=r(arpa) wpa=r(wpa) clrpb=r(clrpb) arpb=r(arpb)
> wpb=r(wpb), /*
> */ reps(1000): sim_iv_clr_15, obs(`nobs') b(`betatrue') mu2k(1.0) rho(0.9)
>
> . su
>
> Variable | Obs Mean Std. Dev. Min Max
> -------------+--------------------------------------------------------
> fsf | 1000 2.035955 .6604241 .5463604 4.59914
> beta2sls | 1000 1.44179 .1285521 .8505121 1.867863
> se2sls | 1000 .1227545 .0347788 .0621307 .3481258
> betaliml | 1000 .8520995 1.848659 -55.81694 1.520073
> seliml | 1000 25.93037 796.7733 .0845153 25196.78
> -------------+--------------------------------------------------------
> betaf1 | 1000 1.015304 .2662472 -.1234837 1.540497
> sef1 | 1000 .4551579 .4818527 .0820186 5.84322
> clrp | 1000 .4620662 .3026814 .0000137 .9511231
> arp | 1000 .36521 .295989 .000068 .9998782
> wp | 1000 .0355579 .1137253 6.65e-23 .9456689
> -------------+--------------------------------------------------------
> clrp2 | 1000 .4868896 .292084 .0000111 .9511231
> arp2 | 1000 .434738 .2942256 .0005332 .9998143
> wp2 | 1000 .0357715 .1145948 2.53e-23 .9488348
> clrp3 | 1000 .4978822 .2895032 .0000214 .9511233
> arp3 | 1000 .4782224 .2936219 .0011465 .999872
> -------------+--------------------------------------------------------
> arF4 | 1000 1.150363 .4478659 .1567113 2.860618
> clrpa | 1000 .3932872 .3109746 9.55e-06 .9511231
> arpa | 1000 .3379416 .2926447 .0000178 .9940948
> wpa | 1000 .0049185 .0369914 0 .7727929
> clrpb | 1000 .2994276 .2921822 7.49e-08 .9511231
> -------------+--------------------------------------------------------
> arpb | 1000 .3025138 .2795973 3.27e-06 .9998758
> wpb | 1000 .2321103 .2962862 7.16e-11 .9958413
> t2sls | 1000 4.047884 1.867733 -.4294076 9.853053
> tliml | 1000 .2971751 .9116821 -.9958801 5.948726
> tf1 | 1000 .4617053 .9595486 -1.005183 6.221221
> -------------+--------------------------------------------------------
> t2slsa | 1000 6.224931 2.370604 .2887236 13.48693
> tlimla | 1000 1.064115 1.350182 -.7798916 8.906771
> tf1a | 1000 1.313411 1.403974 -.7144105 9.269311
> t2slsb | 1000 1.870836 1.388323 -1.147539 6.517281
> tlimlb | 1000 -.4697649 .5260411 -1.484372 2.990681
> -------------+--------------------------------------------------------
> tf1b | 1000 -.3900002 .56222 -1.423636 3.173131
> ttest2sls | 1000 .859 .3481957 0 1
> ttestliml | 1000 .068 .2518719 0 1
> ttestf1 | 1000 .091 .287753 0 1
> clrtest | 1000 .098 .2974634 0 1
> -------------+--------------------------------------------------------
> artest | 1000 .162 .3686352 0 1
> wtest | 1000 .859 .3481957 0 1
> clrtest2 | 1000 .059 .2357426 0 1
> artest2 | 1000 .091 .287753 0 1
> wtest2 | 1000 .864 .3429601 0 1
> -------------+--------------------------------------------------------
> clrtest3 | 1000 .048 .2138732 0 1
> artest3 | 1000 .066 .2484063 0 1
> artest4 | 1000 .121 .3262905 0 1
> ttest2slsa | 1000 .977 .1499783 0 1
> ttestlimla | 1000 .198 .3986916 0 1
> -------------+--------------------------------------------------------
> ttestf1a | 1000 .256 .4366402 0 1
> clrtesta | 1000 .157 .3639828 0 1
> artesta | 1000 .189 .3917045 0 1
> wtesta | 1000 .977 .1499783 0 1
> ttest2slsb | 1000 .448 .4975375 0 1
> -------------+--------------------------------------------------------
> ttestlimlb | 1000 .002 .044699 0 1
> ttestf1b | 1000 .003 .0547174 0 1
> clrtestb | 1000 .256 .4366402 0 1
> artestb | 1000 .23 .4210431 0 1
> wtestb | 1000 .448 .4975375 0 1
>
> Best,
> Tak Wai Chau

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/