RE: st: Multilevel difference modeling with suest

[Cameron McIntosh <cnm100@hotmail.com>]

Peter,

I think the main issue is how exactly to handle the difference scores in the model... my opinion is don't bother with them at all, they're dangerous. I would strongly recommend using polynomial regression instead, for reasons discussed in detail in:
Edwards, J.R. (2002).  Alternatives to difference scores: Polynomial regression analysis and response surface methodology. In F. Drasgow & N. W. Schmitt (Eds.), Advances in measurement and data analysis (pp. 350-400).  San Francisco: Jossey-Bass.
Edwards, J.R. (2001).  Ten difference score myths. Organizational Research Methods, 4, 264-286.http://public.kenan-flagler.unc.edu/faculty/edwardsj/Edwards2001b.pdf

Edwards, J. R., & Cable, D. M.  (2009).  The value of value congruence.  Journal of Applied Psychology, 94(3), 654-677.http://public.kenan-flagler.unc.edu/faculty/edwardsj/EdwardsCable2009.pdf

Edwards, J.R. (2007). Polynomial regression and response surface methodology.  In C. Ostroff & T.A. Judge (Eds.), Perspectives on organizational fit (pp. 361-372).  San Francisco: Jossey-Bass.http://public.kenan-flagler.unc.edu/faculty/edwardsj/Edwards2007.pdf

Klein, G., Jiang, J.J., & Cheney, P. (2009). Resolving Difference Score Issues in Information Systems Research. MIS Quarterly, 33(4), 811-826.

Cafri, G., van den Berg, P., & Brannick, M.T. (2010). What Have the Difference Scores Not Been Telling Us? A Critique of the Use of Self—Ideal Discrepancy in the Assessment of Body Image and Evaluation of an Alternative Data-Analytic Framework. Assessment, 17(3), 361-376.

Venkatesh, V., & Goyal, S. (2010). Expectation Disconfirmation and Technology Adoption: Polynomial Modeling and Response Surface Analysis. MIS Quarterly, 34(2), 281-303.

Cohen, A., Nahum-Shani, I., & Doveh, E. (2010). Further Insight and Additional Inference Methods for Polynomial Regression Applied to the Analysis of Congruence. Multivariate Behavioral Research, 45(5), 828-852.

Shanock, L.R., Baran, B.E., Gentry, W.A., Pattison, S.C., & Heggestad, E.D. (2010). Polynomial Regression with Response Surface Analysis: A Powerful Approach for Examining Moderation and Overcoming Limitations of Difference Scores. Journal of Business and Psychology, 25(4), 543-554.

As for the nesting issue, Taylor linearization or bootstrapping would be reasonable options, as I gather that you are not really interested in explicitly modeling the hierarchical structure and only regard within-cluster correlation as a nuisance parameter. I think you could implement your polynomial regression through -cmp-.
Roodman, D. (2011). Fitting fully observed recursive mixed-process models with cmp. The Stata Journal, 11(2), 159-206.http://www.stata-journal.com/article.html?article=st0224
http://ideas.repec.org/c/boc/bocode/s456882.html
Hope this helps,
Cam

> From: peter.t.goff@vanderbilt.edu
> To: statalist@hsphsun2.harvard.edu
> Subject: st: Multilevel difference modeling with suest
> Date: Sun, 18 Mar 2012 15:42:51 -0500
> 
> Hi All,
> 
> I'm trying to determine the best way to tackle what has been a bit of  
> a slippery problem. My goal is to determine which factors (X) are  
> predictive of the difference between how teachers perceive a  
> principal's leadership (T) and how the principal perceives their own  
> leadership (P). X contains some teacher-level factors (e.g., teacher  
> experience) and some principal-level factors (e.g., principal gender).  
> The literature suggests that the best approach to this problem is to  
> model these equations jointly and then individually test for  
> differences between the coefficients in X. To complicate matters  
> somewhat, teachers are nested within principals so sureg or mvreg  
> can't be used, since neither can accommodate the clustering. I have  
> pursued several suggestions from colleagues and archived statalist  
> posts (e.g., http://www.stata.com/statalist/archive/2009-04/msg01157.html) 
>   that has landed me a bit further from my comfort zone that I'd like.  
> I'd like to present what I have done thus far and hear if anyone has  
> criticism or alternative suggestions.
> 
> reg T X
> 	estimates store t1
> reg P X
> 	estimates store p1
> suest t1 p1, vce(cluster prinid)
> foreach x in X {
> 	test _b[t1_mean:`x'] - _b[p1_mean:`x'] = 0
> }
> 
> In terms of an interpretation, I'd like to use the t1_mean equation  
> from the suest results to make statements about how each of X factors  
> relate to teachers' perceptions of leadership effectiveness; use  
> p1_mean suest results to make statements about how each of X factors  
> relate to the principals' perceptions of their own leadership  
> effectiveness; and use the test results to make statements about how  
> each of X factors relate to the teacher - principal gap. Kind thanks  
> for your thoughts and insights.
> 
> Peter
> peter.t.goff@vanderbilt.edu
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
 		 	   		  
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/