Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: repeated measures ANOVA to MANOVA - revisit [on behalf of Rob Ploutz-Snyder]
From
Ricardo Ovaldia <[email protected]>
To
[email protected]
Subject
RE: st: repeated measures ANOVA to MANOVA - revisit [on behalf of Rob Ploutz-Snyder]
Date
Fri, 3 Feb 2012 11:02:22 -0800 (PST)
Thank you Rob.
This entire thing started because we submitted a manuscript using repeated measures ANOVA and one of the reviewers asked to redo the analysis using MANOVA which he/she feels is better than to adjust the df using Huynh-Feldt or some other method. Although I tend to agree with the premise, I am concerned and confused with the large difference in the p-values that the ANOVA and ANOVA report.
I will like to stay away from -xtmixed- mainly because this is an "accept with minor revisions" paper and I do not want to confuse the reviewers.
Ricardo.
Ricardo Ovaldia, MS
Statistician
Oklahoma City, OK
--- On Fri, 2/3/12, Feiveson, Alan H. (JSC-SK311) <[email protected]> wrote:
> From: Feiveson, Alan H. (JSC-SK311) <[email protected]>
> Subject: RE: st: repeated measures ANOVA to MANOVA - revisit [on behalf of Rob Ploutz-Snyder]
> To: "[email protected]" <[email protected]>
> Date: Friday, February 3, 2012, 9:27 AM
>
> Ricardo..
> I have to be honest... I've abandoned Stata's
> implementation(s) of repeated measures ANOVA entirely, as I
> am rarely able to get it to replicate what I know to be
> accurate with other software (sorry Stata). Likely
> this is due to my confusion about how Stata wants us to
> program something as simple as a repeated measures ANOVA,
> and my personal impatience with that. It isn't as
> straightforward as it should be.
>
> You will find help on Phil Ender's pages at USLA (http://www.ats.ucla.edu/stat/stata/ado/analysis/default.htm)
> where there are several nicely written ado's that'll help
> you understand how to use Stata to do typical ANOVA
> routines.
>
> Having said that, mixed-effects modeling such as implemented
> in Stata's -xtmixed- has distinct advantages over RM ANOVA,
> making the latter a tool that I find myself using less and
> less. If you have a completely balanced factorial
> design and you meet ALL of the assumptions of traditional RM
> ANOVA, then RM ANOVA offers the benefit of a formal F-test
> that doesn’t rely on maximum likelihood estimation.
> It's also somewhat easier to write-up & defend for
> manuscripts, as reviewers (who usually aren't statistician)
> seem to understand "ANOVA" better than "MIXED-EFFECTS
> MODELING," even though that shouldn't be the case.
>
> But if you have imbalance in cell sizes, occasional missing
> data here and there, and the potential for random effects in
> your data, the mixed-modeling (i.e. -xtmixed-) is likely
> going to be your better approach. And... the coding is
> straightforward. And with the new -contrasts- and
> -margins- and -marginsplot- post-estimation routines,
> -xtmixed- functionality is really quite nice for ANVOA-like
> factorial designs. So given those advantages, I
> personally gravitate towards it from the get-go and do my
> best to educate manuscript reviewers along the way.
>
>
> Best,
> Rob
>
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]]
> On Behalf Of Ricardo Ovaldia
> Sent: Friday, February 03, 2012 7:54 AM
> To: [email protected]
> Subject: Re: st: repeated measures ANOVA to MANOVA -
> revisit
>
> I did not get a reply to my question, so here is the
> simplified version:
>
> Using the 23 observations lited below I get very different
> p-values when using repeated measurements ANOVA and MANOVA.
> Why the descrepancy? Am I doing something wrong? Can I use
> -xtmixed- instead (how)?
>
> From MANOVA:
> . gen myconst=1
> . manova m1 m2 m3= myconst, nocons
> . mat c = (1,0,-1\0,1,-1)
> . manovatest mycons, ytransform(c)
> . mat l r(stat)
> p-value=2.519e-07
>
> From repeated ANOVA:
> . reshape long m, i(id) j(method)
> . anova m id method, repeat( method)
> Huynh-Feldt p-value=0.0022.
>
> Data:
> id m1
> m2 m3
>
> 106 22.2 30.6 13.9
>
> 111 26.4 32.2 14.6
>
> 119 23.6 28.9 26.7
>
> 122 27.4 38.0 28.9
>
> 130 17.5 24.5 41.4
>
> 131 18.4 21.5 20.2
>
> 133 28.1 28.1 22.3
>
> 135 33.5 38.5 29.9
>
> 140 18.9 25.7 15.1
>
> 144 21.2 28.3 37.0
>
> 149 18.8 25.6 15.5
>
> 152 22.4 31.5 28.5
>
> 153 21.5 28.6 22.5
>
> 158 27.9 37.6 37.2
>
> 167 30.1 42.3 24.0
>
> 168 28.5 36.9 32.4
>
> 171 23.5 36.7 27.8
>
> 176 24.6 24.5 25.8
>
> 180 29.4 30.3 15.2
>
> 188 23.2 24.6
> 9.1
>
> 191 25.7 31.7 31.2
>
> 192 20.1 19.7
> 8.2
>
> 194 23.4 27.7
> 6.0
>
> Thank you,
> Ricardo
>
> --- On Thu, 2/2/12, Ricardo Ovaldia <[email protected]>
> wrote:
>
> > From: Ricardo Ovaldia <[email protected]>
> > Subject: Re: st: repeated measures ANOVA to MANOVA -
> revisit
> > To: [email protected]
> > Date: Thursday, February 2, 2012, 8:18 AM
> > Using the 23 observations below I get
> > very different (and concerning) p-values when I use
> repeated
> > measurements ANOVA and MANOVA.
> > From MANOVA:
> > . gen myconst=1
> > . manova m1 m2 m3= myconst, nocons
> > <Output suppressed>
> > . mat c = (1,0,-1\0,1,-1)
> > . manovatest mycons, ytransform(c)
> > <Output suppressed>
> > . mat l r(stat)
> >
> > r(stat)[4,6]
> > statistic
> > F df1
> > df2 pvalue
> > exact
> > Wilks .23525472 34.132474
> > 2
> > 21 2.519e-07
> > 1
> > Pillai .76474528 34.132474
> > 2
> > 21 2.519e-07
> > 1
> > Lawley 3.2507118 34.132474
> > 2
> > 21 2.519e-07
> > 1
> > Roy 3.2507118 34.132474
> > 2
> > 21 2.519e-07
> > 1
> >
> > From repeated ANOVA:
> > . drop myconst
> > . reshape long m, i(id) j(method)
> > <Output suppressed>
> >
> > . anova m id method, repeat( method)
> > <Top of output suppressed>
> > …
> > Between-subjects error term: id
> >
> > Levels: 23
> > (22 df)
> > Lowest b.s.e. variable: id
> >
> > Repeated variable: method
> >
> >
> > Huynh-Feldt epsilon
> > = 0.6357
> >
> >
> > Greenhouse-Geisser
> > epsilon = 0.6174
> >
> >
> > Box's conservative
> > epsilon = 0.5000
> >
> >
> >
> > ------------ Prob
> > > F ------------
> >
> > Source | df
> > F Regular H-F
> > G-G Box
> >
> >
> -----------+----------------------------------------------------
> >
> > method | 2
> > 10.12 0.0002 0.0020 0.0022 0.0043
> >
> > Residual | 44
> >
> >
> ----------------------------------------------------------------
> >
> > MANOVA reports a p-value of 2.519e-07, whereas the
> > Huynh-Feldt p-value from ANOVA is 0.0022.
> > Any idea why they are so different? Am I doing
> something
> > wrong?
> >
> > Thank you,
> > Ricardo
> >
> > Data:
> > id m1
> > m2 m3
> >
> > 106 22.2 30.6 13.9
> >
> > 111 26.4 32.2 14.6
> >
> > 119 23.6 28.9 26.7
> >
> > 122 27.4 38.0 28.9
> >
> > 130 17.5 24.5 41.4
> >
> > 131 18.4 21.5 20.2
> >
> > 133 28.1 28.1 22.3
> >
> > 135 33.5 38.5 29.9
> >
> > 140 18.9 25.7 15.1
> >
> > 144 21.2 28.3 37.0
> >
> > 149 18.8 25.6 15.5
> >
> > 152 22.4 31.5 28.5
> >
> > 153 21.5 28.6 22.5
> >
> > 158 27.9 37.6 37.2
> >
> > 167 30.1 42.3 24.0
> >
> > 168 28.5 36.9 32.4
> >
> > 171 23.5 36.7 27.8
> >
> > 176 24.6 24.5 25.8
> >
> > 180 29.4 30.3 15.2
> >
> > 188 23.2 24.6
> > 9.1
> >
> > 191 25.7 31.7 31.2
> >
> > 192 20.1 19.7
> > 8.2
> >
> > 194 23.4 27.7
> > 6.0
> >
> >
> >
> > Ricardo Ovaldia, MS
> > Statistician
> > Oklahoma City, OK
> >
> >
> > --- On Wed, 2/1/12, Ricardo Ovaldia <[email protected]>
> > wrote:
> >
> > > From: Ricardo Ovaldia <[email protected]>
> > > Subject: Re: st: repeated measures ANOVA to
> MANOVA
> > > To: [email protected]
> > > Date: Wednesday, February 1, 2012, 9:18 AM
> > > Never mind. Thank you. I found the
> > > answer on page 359 of the manual.
> > > I am now concerned because the pvalue from MANOVA
> is so
> > much
> > > smaller than the Huynh-Feldt corrected p-value
> > >
> > > Thank you again,
> > > Ricardo
> > >
> > > Ricardo Ovaldia, MS
> > > Statistician
> > > Oklahoma City, OK
> > >
> > >
> > > --- On Wed, 2/1/12, Ricardo Ovaldia <[email protected]>
> > > wrote:
> > >
> > > > From: Ricardo Ovaldia <[email protected]>
> > > > Subject: st: repeated measures ANOVA to
> MANOVA
> > > > To: "Statalist" <[email protected]>
> > > > Date: Wednesday, February 1, 2012, 8:50 AM
> > > > I have data on 23 patients that were
> > > > evaluated using three competing medical
> methods.
> > I
> > > used
> > > > repeated measures ANOVA and reported the
> > Huynh-Feldt
> > > > corrected p-value. A reviewer suggested that
> it
> > would
> > > be
> > > > better to do a MANOVA. However, when I try
> > this,
> > > Stata
> > > > reports the error:
> > > >
> > > > . manova m1 m2 m3=id
> > > > matrix not positive definite
> > > > insufficient residual degrees of freedom for
> this
> > > > multivariate model
> > > >
> > > > Any help will be appreciated.
> > > > Ricardo
> > > >
> > > > Here is the data:
> > > > . cl id m1 m2 m3
> > > >
> > > > id
> > > > m1 m2
> > > > m3
> > > > 1.
> > > > 106 22.2 30.6 13.9
> > > > 2.
> > > > 111 26.4 32.2 14.6
> > > > 3.
> > > > 119 23.6 28.9 26.7
> > > > 4.
> > > > 122 27.4 38.0 28.9
> > > > 5.
> > > > 130 17.5 24.5 41.4
> > > > 6.
> > > > 131 18.4 21.5 20.2
> > > > 7.
> > > > 133 28.1 28.1 22.3
> > > > 8.
> > > > 135 33.5 38.5 29.9
> > > > 9.
> > > > 140 18.9 25.7 15.1
> > > > 10.
> > > > 144 21.2 28.3 37.0
> > > > 11.
> > > > 149 18.8 25.6 15.5
> > > > 12.
> > > > 152 22.4 31.5 28.5
> > > > 13.
> > > > 153 21.5 28.6 22.5
> > > > 14.
> > > > 158 27.9 37.6 37.2
> > > > 15.
> > > > 167 30.1 42.3 24.0
> > > > 16.
> > > > 168 28.5 36.9 32.4
> > > > 17.
> > > > 171 23.5 36.7 27.8
> > > > 18.
> > > > 176 24.6 24.5 25.8
> > > > 19.
> > > > 180 29.4 30.3 15.2
> > > > 20.
> > > > 188 23.2 24.6
> > > > 9.1
> > > > 21.
> > > > 191 25.7 31.7 31.2
> > > > 22.
> > > > 192 20.1 19.7
> > > > 8.2
> > > > 23.
> > > > 194 23.4 27.7
> > > > 6.0
> > > >
> > > >
> > > > Ricardo Ovaldia, MS
> > > > Statistician
> > > > Oklahoma City, OK
> > > > *
> > > > * For searches and help try:
> > > > * http://www.stata.com/help.cgi?search
> > > > * http://www.stata.com/support/statalist/faq
> > > > * http://www.ats.ucla.edu/stat/stata/
> > > >
> > >
> > > *
> > > * For searches and help try:
> > > * http://www.stata.com/help.cgi?search
> > > * http://www.stata.com/support/statalist/faq
> > > * http://www.ats.ucla.edu/stat/stata/
> > >
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/help.cgi?search
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/