st: Stumped...xtmixed and ANOVA F-stats not agreeing for balanced design

[Jared Saletin <jsaletin@berkeley.edu>]

Dear all,

Thanks for all the help in previous messages. This thoughtful user community makes learning all the more enjoyable, so  thank you.

Hope some of the experts out there can help me  get passed a stumping issue, comparing xtmixed with ANOVA.

I'm running a Within-Subject model with 2 factors (a, b). The F-stats produced by the two methods are not agreeing, despite having balanced data.

Following the suggestions made in this message: http://www.stata.com/statalist/archive/2010-03/msg01340.html

I'm parameterizing my ANOVA as:

anova  y s a/s#a b/s#b a#b/ 

which yields

                           Number of obs =      66     R-squared     =  0.9746
                           Root MSE      = .052335     Adj R-squared =  0.9175

                  Source |  Partial SS    df       MS           F     Prob > F
              -----------+----------------------------------------------------
                   Model |  2.10365629    45  .046747918      17.07     0.0000
                         |
                       s |  .202683096    10   .02026831       1.58     0.1840
                       a |  .358787807     2  .179393903      13.99     0.0002
                     s#a |  .256516955    20  .012825848   
              -----------+----------------------------------------------------
                       b |   1.1109981     1   1.1109981      76.02     0.0000
                     s#b |  .146140374    10  .014614037   
              -----------+----------------------------------------------------
                     a#b |  .028529959     2  .014264979       5.21     0.0151
                         |
                Residual |  .054779806    20   .00273899   
              -----------+----------------------------------------------------
                   Total |   2.1584361    65  .033206709   


and then estimating the same model with xtmixed, followed by anovalator for the fixed effects:

xtmixed y a##b || s: || s: R.a || s: R.b
anovalator a b, main 2way fratio

yielding:
Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log restricted-likelihood =  50.636505  
Iteration 1:   log restricted-likelihood =  50.790265  
Iteration 2:   log restricted-likelihood =  50.792902  
Iteration 3:   log restricted-likelihood =  50.792903  

Computing standard errors:

Mixed-effects REML regression                   Number of obs      =        66
Group variable: s                               Number of groups   =        11

                                                Obs per group: min =         6
                                                               avg =       6.0
                                                               max =         6


                                                Wald chi2(5)       =    120.97
Log restricted-likelihood =  50.792903          Prob > chi2        =    0.0000

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           a |
          2  |  -.1066975   .0371271    -2.87   0.004    -.1794653   -.0339297
          3  |   .0104766   .0371271     0.28   0.778    -.0622911    .0832444
             |
         2.b |  -.2348552   .0340381    -6.90   0.000    -.3015687   -.1681417
             |
         a#b |
        2 2  |   -.083192   .0316799    -2.63   0.009    -.1452834   -.0211005
        3 2  |   .0092982   .0316799     0.29   0.769    -.0527932    .0713897
             |
       _cons |   .8529769   .0318999    26.74   0.000     .7904542    .9154996
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
s: Identity                  |
                   sd(_cons) |   2.70e-08   .0000919             0           .
-----------------------------+------------------------------------------------
s: Identity                  |
                     sd(R.a) |   .0694361   .0134607       .047487    .1015305
-----------------------------+------------------------------------------------
s: Identity                  |
                     sd(R.b) |   .0601026    .014489      .0374708    .0964037
-----------------------------+------------------------------------------------
                sd(Residual) |   .0525352   .0083535      .0384683     .071746
------------------------------------------------------------------------------
LR test vs. linear regression:       chi2(3) =    15.67   Prob > chi2 = 0.0013

Note: LR test is conservative and provided only for reference.

. anovalator a b, main 2way fratio

anovalator main-effect for a  
chi2(2) = 28.928213   p-value = 5.228e-07
scaled as F-ratio = 14.464106

anovalator main-effect for b  
chi2(1) = 81.709563   p-value = 1.576e-19
scaled as F-ratio = 81.709563

anovalator two-way interaction for a#b  
chi2(2) = 10.337154   p-value = .00569266
scaled as F-ratio = 5.1685769

---

Despite the data being balanced, the F-statistics generated from xtmixed and ANOVA still don't match.

I presume it has something to do with the standard error and therefore CI for the constant in the random-effects model not being estimated. I'm not sure why this is the case. I've run larger models, even unbalanced models where the random effects are estimated fine.

Is there something inherently in the data that would cause that SE estimate to fail?

Thanks for any suggestions you all may have.

Cheers,
Jared




*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/