Hi Brent, I'm afraid I can't provide much intuition here since it
would probably require familiarity with the data. You could verify
the result using likelihood tests, or interaction terms and compare
the result to a suest result to see if your result is reasonable. Of
course you would have to do this comparison without the survey
estimators I think but it could help see if the suest result is
reasonable. You should get the same results for interaction terms as
you do for suest, for example note the p-value for the coefficient on
the interaction terms for weight and headroom is the same as that when
you test the coefficients after suest.
Tim
. sysuse auto.dta
(1978 Automobile Data)
. gen mpgcat=cond(1, mpg>=20, 0)
. logit foreign weight headroom if mpg<20
Iteration 0: log likelihood = -14.354071
Iteration 1: log likelihood = -8.6195982
Iteration 2: log likelihood = -6.3389471
Iteration 3: log likelihood = -5.776879
Iteration 4: log likelihood = -5.655399
Iteration 5: log likelihood = -5.6441813
Iteration 6: log likelihood = -5.644033
Iteration 7: log likelihood = -5.644033
Logistic regression Number of obs = 35
LR chi2(2) = 17.42
Prob > chi2 = 0.0002
Log likelihood = -5.644033 Pseudo R2 = 0.6068
------------------------------------------------------------------------------
foreign | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
weight | -.0094147 .0054628 -1.72 0.085 -.0201216 .0012922
headroom | -.1793897 .869445 -0.21 0.837 -1.883471 1.524691
_cons | 29.9695 17.37471 1.72 0.085 -4.084307 64.02331
------------------------------------------------------------------------------
. est store m1
. logit foreign weight headroom if mpg>=20
Iteration 0: log likelihood = -26.711343
Iteration 1: log likelihood = -20.190873
Iteration 2: log likelihood = -19.24142
Iteration 3: log likelihood = -19.157028
Iteration 4: log likelihood = -19.156039
Iteration 5: log likelihood = -19.156039
Logistic regression Number of obs = 39
LR chi2(2) = 15.11
Prob > chi2 = 0.0005
Log likelihood = -19.156039 Pseudo R2 = 0.2829
------------------------------------------------------------------------------
foreign | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
weight | -.0032841 .0011223 -2.93 0.003 -.0054837 -.0010846
headroom | .1883558 .6371348 0.30 0.768 -1.060405 1.437117
_cons | 7.107416 3.061682 2.32 0.020 1.106629 13.1082
------------------------------------------------------------------------------
. est store m2
. suest m1 m2
Simultaneous results for m1, m2
Number of obs = 74
------------------------------------------------------------------------------
| Robust
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
m1 |
weight | -.0094147 .004095 -2.30 0.022 -.0174408 -.0013885
headroom | -.1793897 .4468522 -0.40 0.688 -1.055204 .6964245
_cons | 29.9695 13.53867 2.21 0.027 3.434191 56.50481
-------------+----------------------------------------------------------------
m2 |
weight | -.0032841 .0010726 -3.06 0.002 -.0053864 -.0011819
headroom | .1883558 .6625353 0.28 0.776 -1.110189 1.486901
_cons | 7.107416 3.342944 2.13 0.033 .555367 13.65947
------------------------------------------------------------------------------
. test [m1]weight=[m2]weight
( 1) [m1]weight - [m2]weight = 0
chi2( 1) = 2.10
Prob > chi2 = 0.1476
. test [m1]headroom=[m2]headroom
( 1) [m1]headroom - [m2]headroom = 0
chi2( 1) = 0.21
Prob > chi2 = 0.6454
. xi: logit foreign i.mpgcat*weight i.mpgcat*headroom, vce(robust)
i.mpgcat _Impgcat_0-1 (naturally coded; _Impgcat_0 omitted)
i.mpgcat*weight _ImpgXweigh_# (coded as above)
i.mpgcat*head~m _ImpgXheadr_# (coded as above)
note: _Impgcat_1 dropped because of collinearity
Iteration 0: log pseudolikelihood = -45.03321
Iteration 1: log pseudolikelihood = -29.341451
Iteration 2: log pseudolikelihood = -26.225121
Iteration 3: log pseudolikelihood = -25.164491
Iteration 4: log pseudolikelihood = -24.856965
Iteration 5: log pseudolikelihood = -24.802901
Iteration 6: log pseudolikelihood = -24.800082
Iteration 7: log pseudolikelihood = -24.800072
Logistic regression Number of obs = 74
Wald chi2(5) = 15.57
Prob > chi2 = 0.0082
Log pseudolikelihood = -24.800072 Pseudo R2 = 0.4493
------------------------------------------------------------------------------
| Robust
foreign | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
weight | -.0094147 .0040949 -2.30 0.021 -.0174406 -.0013888
_ImpgXweig~1 | .0061305 .0042331 1.45 0.148 -.0021661 .0144272
_Impgcat_1 | -22.86209 13.94489 -1.64 0.101 -50.19356 4.469389
headroom | -.1793897 .4468517 -0.40 0.688 -1.055203 .6964235
_ImpgXhead~1 | .3677455 .7991429 0.46 0.645 -1.198546 1.934037
_cons | 29.9695 13.53826 2.21 0.027 3.434993 56.50401
------------------------------------------------------------------------------
Tim
On Wed, Jun 25, 2008 at 6:42 PM, Brent Fulton <[email protected]> wrote:
> Hi Tim, thank for your response. I agree the covariance term could matter so
> I used your suggested command -matlist e(V)-
>
> The cov([m1a]hispanic, [m1b]hispanic) was 0.12, so standard error of the
> difference 0.017 becomes quite small (0.011), making the p-value of 0.0991
> make sense.
>
> se of difference: sqrt( (0.347001)^2 + (0.34728)^2 - 2(0.120451))= 0.011
>
> Now I realize the covariance term really mattered, but I don't know quite
> how to explain it. The standard error of each estimate was quite wide at
> 0.347, yet the standard error of the estimate of the difference between
> these estimators was only 0.011. The models were very similar; only one
> independent variable was added to the second model.
>
> I'd appreciate any intuition on how to explain how the standard error of the
> difference is very small relative to the standard error of each parameter
> estimate.
>
> Thanks,
> Brent
>
>
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Tim Wade
> Sent: Wednesday, June 25, 2008 2:38 PM
> To: [email protected]
> Subject: Re: st: testing equality of parameter estimates across two
> models--suest followed by test
>
> Brent,
>
> I've never used suest with survey estimators and I'm not sure how the
> F-statistic is obtained. But I don't think you can rely only on the
> standard errors for the coefficients to assess the precision of the
> estimator. Unless the two models are independent you need to take into
> account the cross-model covariance, which you can list after you
> "suest" your models using "matlist e(V)". If they have a positive
> covariance this will reduce the variance of the (a-b) estimate since:
> Var(a-b)=Var(a)+Var(b)-2*cov(a, b)
>
>
> for example:
>
>
> . sysuse auto.dta
> (1978 Automobile Data)
>
> . logistic foreign mpg
>
> Logistic regression Number of obs =
> 74
> LR chi2(1) =
> 11.49
> Prob > chi2 =
> 0.0007
> Log likelihood = -39.28864 Pseudo R2 =
> 0.1276
>
> ----------------------------------------------------------------------------
> --
> foreign | Odds Ratio Std. Err. z P>|z| [95% Conf.
> Interval]
> -------------+--------------------------------------------------------------
> --
> mpg | 1.173232 .0616972 3.04 0.002 1.058331
> 1.300608
> ----------------------------------------------------------------------------
> --
>
> . est store m1
>
> . logistic foreign mpg headroom
>
> Logistic regression Number of obs =
> 74
> LR chi2(2) =
> 13.39
> Prob > chi2 =
> 0.0012
> Log likelihood = -38.34058 Pseudo R2 =
> 0.1486
>
> ----------------------------------------------------------------------------
> --
> foreign | Odds Ratio Std. Err. z P>|z| [95% Conf.
> Interval]
> -------------+--------------------------------------------------------------
> --
> mpg | 1.139888 .0623668 2.39 0.017 1.023977
> 1.268919
> headroom | .598804 .227678 -1.35 0.177 .2842103
> 1.261623
> ----------------------------------------------------------------------------
> --
>
> . est store m2
>
> . suest m1 m2
>
> Simultaneous results for m1, m2
>
> Number of obs =
> 74
>
> ----------------------------------------------------------------------------
> --
> | Robust
> | Coef. Std. Err. z P>|z| [95% Conf.
> Interval]
> -------------+--------------------------------------------------------------
> --
> m1 |
> mpg | .1597621 .0514512 3.11 0.002 .0589197
> .2606046
> _cons | -4.378866 1.181215 -3.71 0.000 -6.694005
> -2.063727
> -------------+--------------------------------------------------------------
> --
> m2 |
> mpg | .1309298 .0546012 2.40 0.016 .0239135
> .2379461
> headroom | -.512821 .3168964 -1.62 0.106 -1.133927
> .1082846
> _cons | -2.284788 1.838875 -1.24 0.214 -5.888917
> 1.319342
> ----------------------------------------------------------------------------
> --
>
> . matlist e(V)
>
> | m1 | m2
> | mpg _cons | mpg headroom _cons
> -------------+----------------------+---------------------------------
> m1 | |
> mpg | .0026472 |
> _cons | -.0590473 1.395269 |
> -------------+----------------------+---------------------------------
> m2 | |
> mpg | .0026541 -.0605161 | .0029813
> headroom | .0012744 -.0545147 | .0064742 .1004233
> _cons | -.0627595 1.580435 | -.0860442 -.4539693 3.381462
>
> . test [m1]mpg=[m2]mpg
>
> ( 1) [m1]mpg - [m2]mpg = 0
>
> chi2( 1) = 2.60
> Prob > chi2 = 0.1071
>
> . scalar num=0.1597621-0.1309298
>
> . scalar denom=0.0026472+0.0029813-(2*0.0026541)
>
> . di chi2tail(1, ((num)/sqrt(denom))^2)
> .10717545
>
> Hope this helps, Tim
>
>
> On Wed, Jun 25, 2008 at 2:51 PM, Brent Fulton <[email protected]> wrote:
>> Hi,
>>
>> I am using Stata 9.2 and wanted to test whether the parameter estimate for
>> the variable "hispanic" was statistically different in model 1 as compared
>> to model 2. I assume that -suest- followed by -test- is the appropriate
>> procedure, but it is giving me puzzling results. How is it possible when
> the
>> estimates only differ by 0.017 that the p-value of the Wald test is
>> remarkably low at 0.0991? It must be the case that the standard error of
> the
>> 0.017 statistic is very low, but I'm not sure how that's possible given
> that
>> the standard errors of the parameter estimates in both models were 0.347.
>>
>> I'd appreciate your advice.
>>
>> code:
>> svy, subpop(if sample1==1): logit adhd_d $ind1a `geodummies'
>> estimates store m1a
>> svy, subpop(if sample1==1): logit adhd_d $ind1b `geodummies'
>> estimates store m1b
>> suest m1a m1b, svy
>> test [m1a]hispanic=[m1b]hispanic
>>
>> output:
>> m1a--hispanic parameter, (se), (p-value): -0.310 (0.347) (0.372)
>> m1b--hispanic parameter, (se), (p-value): -0.327 (0.347) (0.346)
>>
>> Adjusted Wald test (testing equality of hispanic parameter estimates)
>> F( 1, 78762) = 2.72
>> Prob > F = 0.0991
>>
>>
>> Thanks,
>> Brent Fulton
>>
>> p.s. note that is a similar question to the link below, but no answer was
>> posted
>> http://www.stata.com/statalist/archive/2007-10/msg00870.html
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/support/faqs/res/findit.html
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
