RE: st: Does MANCOVA suit my research question?

[Cameron McIntosh <cnm100@hotmail.com>]

Sascha,
Well, you can include interactions in the (M)ANCOVA to test the assumption of homogeneity of regression. If they are non-significant, this reduces to the standard MANCOVA model. If they are significant, you can leave them in. They will make things a bit trickier to interpret, but interactions do not preclude use of MANCOVA:
Huitema, B. (2011). The Analysis of Covariance and Alternatives: Statistical Methods for Experiments, Quasi-Experiments, and Single-Case Studies (2nd ed.). Hoboken, NJ: Wiley.
Cardinal, R.N., & Aitken, M.R.F. (2006). ANOVA for the behavioral science researcher. Mahwah, NJ: Erlbaum.

Rutherford, A. (2001). Introducing ANOVA and ANCOVA: a GLM approach. Thousand Oaks, CA: Sage.
However, I still stand by my residual (latent) change score suggestion. :) 
Cam
----------------------------------------
> Date: Mon, 16 Jan 2012 15:32:47 +0100
> Subject: Re: st: Does MANCOVA suit my research question?
> From: saschastruys@gmail.com
> To: statalist@hsphsun2.harvard.edu
>
> Cameron,
>
> Thank you again for your effort and helpfull answers.
> From your reference (Cribbie & Jamieson, 2004), I understood that
> MANCOVA with pretest scores as covariate would be the most suitable to
> answer the research question as there is indeed a floor effect on the
> posttest-scores (and on the pre-test scores for that matter). As I am
> not educated in SEM, that would not be an option for me at the moment.
>
> I am in doubt about two assumptions of MANCOVA though:
> -1 Homogeneity of the regression hyperplanes.
> In order for MANCOVA to be appropriate, the regression lines of
> the covariates should be the same for all groups. However, we do not
> want to compare groups, we want to describe the association with
> continuous IV on the DV combined and seperate. We DO have groups in
> the model though (female/male) (partner vs. no partner) etc. Does this
> assumption also hold for our research question? The way I see it, is
> that we have only one group (the whole sample) and so there is only
> one regression line, so equality is perfect. Or does this assumption
> also hold for continuous variables (i.e. covariates should have the
> same relationship with the DV for every value of the contiuous
> predictors)? If so, how would you test for it (maybe interaction terms
> of all predictor variables with all covariates)?
>
> -2 Significant linear relationships between de covariates and the DV's.
> Some of the covariates in the model do not have a significant
> relationship with the DV. Does that mean that this assumption is
> violated? If so, is there a way to adjust for this?
>
> Thanks again for your consideration,
>
> Sascha
>
> Cribbie, R.A., & Jamieson, J. (2004). Decreases in Posttest Variance
> and The Measurement of Change. Methods of Psychological Research,
> 9(1), 37-55. http://www.dgps.de/fachgruppen/methoden/mpr-online/issue22/mpr124_10.pdf
>
> 2012/1/14 Cameron McIntosh :
> > Sascha,
> >
> > I believe that Pillai's trace in MANCOVA gives you the combined "net" effects on the outcomes, yes -- the sums of the explained variances on the DVs with respect to each IV (and each covariates as well).
> > But I would think that what you really want are the effects of the IVs on the change from baseline to follow-up. A change score model would help you show that.
> >
> > Cam
> >
> >> Date: Sat, 14 Jan 2012 14:10:06 +0100
> >> Subject: Re: st: Does MANCOVA suit my research question?
> >> From: saschastruys@gmail.com
> >> To: statalist@hsphsun2.harvard.edu
> >>
> >> Dear Cameron, Statalist users,
> >>
> >> Thank you very much for your insightful response.
> >> Your Answer provides me with insights and references I would have
> >> surely overlooked otherwise.
> >> I will look into the (latent) residualized change score model and hope
> >> to use it (the outcomes are in fact scales consisting of 11 and 8
> >> questions).
> >>
> >> There is still something that is not yet clear to me though.
> >> Can MANCOVA be used to examine the association of two continous
> >> variables without comparing groups?
> >> Does the multivariate statistic (e.g. Pillai's Trace) stand for the
> >> amount of variance that an (continuous) IV explains in the outcome
> >> measures when the variance of all other IV in the model is adjusted
> >> for? If not, is there an other way to show if the association of the
> >> continuous predictors with the continous outcomes combined is
> >> statisticlly significant (i.e. some kind of test).
> >>
> >> Thank you for your consideration,
> >>
> >> S.Y. Struijs
> >>
> >> 2012/1/14 Cameron McIntosh :
> >> > I think you could accomplish the same thing with a residualized change score model, which might be a bit easier/elegant to present and interpret. Just do a multivariate regression of both outcomes on all predictors (including the baseline measure) -- too bad you don't have a longer follow-up period that would allow you to observe and model the longer-term trend in your outcome.If you have multiple observed indicators for your continuous outcome (i.e., if it is some form of summary scale score), then you could also control for measurement error by upgrading to a latent residualized change score model:
> >> >
> >> > Cribbie, R.A., & Jamieson, J. (2004). Decreases in Posttest Variance and The Measurement of Change. Methods of Psychological Research, 9(1), 37-55. http://www.dgps.de/fachgruppen/methoden/mpr-online/issue22/mpr124_10.pdf
> >> >
> >> > Ferrando, P.J., & Anguiano-Carrasco, C. (2011). A Structural Equation Model at the Individual and Group Level for Assessing Faking-Related Change. Structural Equation Modeling, 18(1), 91-109.
> >> >
> >> > Leonhart, R., Wirtz, M., & Bengel, J. (2008). Measuring effect sizes using manifest versus latent variables: consequences and implications for research. International Journal of Rehabilitation Research, 31(3), 207-216.
> >> >
> >> > Vautier, S., Steyer, R., & Boomsma, A. (2008). The true-change model with individual method effects: Reliability issues. British Journal of Mathematical and Statistical Psychology, 61(2), 379–399.
> >> >
> >> > Steyer, R., Eid, M., & Schwenkmezger, P. (1997). Modeling true intraindividual change: True change as a latent variable. Methods of Psychological Research–Online, 2, 21–33.
> >> >
> >> > Mun, E.Y., von Eye, A., & White, H.R. (2009). An SEM Approach for the Evaluation of Intervention Effects Using Pre-Post-Post Designs. Structural Equation Modeling, 16(2), 315-337.
> >> > Cam
> >> >
> >> > ----------------------------------------
> >> >> Date: Fri, 13 Jan 2012 20:08:23 +0100
> >> >> Subject: st: Does MANCOVA suit my research question?
> >> >> From: saschastruys@gmail.com
> >> >> To: statalist@hsphsun2.harvard.edu
> >> >>
> >> >> Dear Statalist users,
> >> >>
> >> >> I have doubts regarding the suitability of the statistical model I am
> >> >> currently using to answer our research question. I am using Stata
> >> >> 11.2.
> >> >>
> >> >> The following variables are involved:
> >> >> - Two continuous outcome measures measured on two occasions (baseline
> >> >> and 1-year follow-up) Out1, Out2, Cov1, Cov2
> >> >> - Four continuous predictor variables, Pre1, Pre2, Pre3, Pre4
> >> >> - Four continuous, one ordinal and three nominal covariates, Cov3,
> >> >> Cov4, Cov5, Cov6, Cov7(ordinal), Cov8(nominal), Cov9(nominal),
> >> >> Cov10(nominal)
> >> >> (these are demographics and measures that are associated with the
> >> >> outcome measure)
> >> >>
> >> >> We want to estimate the association of the four predictor variables
> >> >> (Pre1 to 4) with the two outcome variables (Out1 & 2) assessed at the
> >> >> 1 year follow-up,
> >> >> adjusting for the same measure assessed at baseline (Cov1 & 2) and
> >> >> other covariates (Cov3 to 10).
> >> >> Next to that we predict that the four predictor variables will be
> >> >> stronger associated with outcome 1 versus outcome 2.
> >> >>
> >> >> I took the following steps: first, I build a MANOVA model testing for
> >> >> the overall significance of the model
> >> >> and the overall significance of each predictor variable on both
> >> >> outcome measures combined.
> >> >> Second, I performed multivariate regression analysis to obtain the
> >> >> coefficients of the predictor variables for both outcomes.
> >> >> Third, I performed four F-tests to test the hypothesis that the
> >> >> predictor variables are stronger associated with outcome 1 versus
> >> >> outcome 2.
> >> >>
> >> >> I used the following syntax:
> >> >>
> >> >> . manova Out1 Out2 = c.Cov1 c.Cov2 c.Cov3 c.Cov4 c.Cov5 c.Cov6 Cov7
> >> >> Cov8 Cov9 Cov10 c.Pre1 c.Pre2 c.Pre3 c.Pre4
> >> >> . mvreg
> >> >> . test [Out1]Pre1 = [Out2]Pre1
> >> >> . test [Out1]Pre1 = [Out2]Pre2
> >> >> . test [Out1]Pre1 = [Out2]Pre3
> >> >> . test [Out1]Pre1 = [Out2]Pre4
> >> >>
> >> >> My questions are:
> >> >> Is this a suitable statistical model to answer our research question?
> >> >> Does the multivariate statistic for Pre1 (continuous variable) and its
> >> >> corresponding significant (p<0.05) F-ratio
> >> >> state that the overall association of Pre1 with the combined outcome
> >> >> measures, adjusted for covariates and other predictors, is
> >> >> statistically significant at the p<0.5 level?
> >> >>
> >> >> Does this model automatically adjusts the means of the outcome measures,
> >> >> so that an overall significance of a predictor demonstrate that the
> >> >> effect of that predictor is significant
> >> >> given that the sample would have the same scores on all other
> >> >> covariates and predictors? Would the method of analysis then be
> >> >> MANCOVA?
> >> >>
> >> >> Thanks for your consideration,
> >> >>
> >> >> S.Y. Struijs
> >> >> *
> >> >> * For searches and help try:
> >> >> * http://www.stata.com/help.cgi?search
> >> >> * http://www.stata.com/support/statalist/faq
> >> >> * http://www.ats.ucla.edu/stat/stata/
> >> >
> >> > *
> >> > *   For searches and help try:
> >> > *   http://www.stata.com/help.cgi?search
> >> > *   http://www.stata.com/support/statalist/faq
> >> > *   http://www.ats.ucla.edu/stat/stata/
> >>
> >> *
> >> *   For searches and help try:
> >> *   http://www.stata.com/help.cgi?search
> >> *   http://www.stata.com/support/statalist/faq
> >> *   http://www.ats.ucla.edu/stat/stata/
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
 		 	   		  
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/