Re: st: Multilevel difference modeling with suest

[Austin Nichols <austinnichols@gmail.com>]

Peter Goff <peter.t.goff@vanderbilt.edu>:
First, read
 help _mtest
to see why you can't test one coef after another without distorting
size of tests.

However, I don't see how testing for nonzero differences in
coefficients is helpful in your case.
Imagine T is P times 2 plus noise; coefs in the T eq are roughly twice as large.
I.e. T evaluate bad P twice as harshly and good P twice as kindly on
average, but otherwise rankings are identical in expectation.
What does it mean that you can reject z=y then?
The equations are the same in any real sense.
Using rank or standardizing need not solve the problem of comparability.

Maybe you can
  cap ssc inst ivreg2
  ivreg2 P (T=X*), cl(Pid)
where the null of the overid test is that X has no impact on P except
via T; if there is an independent effect of  an X on P you should
reject the null.
This is a highly nonstandard use of an overid test, and I have not
thought through the implications or assumptions.
Not at all in the -mvreg- frame, for sure.

Another thing to worry about: do X variables also affect skewness of T
evaluations?
The conditional mean may not be the best measure of a typical
teacher's evaluation.

On Mon, Mar 19, 2012 at 4:50 PM, Peter Goff <peter.t.goff@vanderbilt.edu> wrote:
> Thanks for your thoughts on the modeling the Teacher-Principal (T-P)
> difference. As it turns out, polynomial regression is a great method for
> modeling differences between variables when the differenced quantity of
> interest is an independent variable. However, in the situation below I'd
> like to use a vector of independent variables (X) to model/predict the T-P
> difference as the dependent variable.
>
> T = a + zX + e
> P= b + yX + u
> (T-P) = (a-b) + (z-y)X + (e+u)
>
> Any comments on whether the method I outline is appropriate for multi-level
> modeling of seemingly unrelated regression and whether I have identified the
> appropriate approach to test for non-zero difference between coefficients is
> kindly appreciated. That is, will this approach provide the appropriate
> standard errors to test:
>
> z = 0
> y = 0
> (z-y) = 0
>
> To be clear, principal self-evaluations (P) are constant within principals
> but vary between principals. Teacher evaluations of principals vary within
> and between principals. Some of the X variables are teacher-level and vary
> both between and within principals; others are principal-level variables and
> only vary between groups.
>
> Kind thanks,
> ~Peter
> peter.t.goff@vanderbilt.edu
>
>
>> Hi All,
>>
>> I'm trying to determine the best way to tackle what has been a bit of
>> a slippery problem. My goal is to determine which factors (X) are
>> predictive of the difference between how teachers perceive a
>> principal's leadership (T) and how the principal perceives their own
>> leadership (P). X contains some teacher-level factors (e.g., teacher
>> experience) and some principal-level factors (e.g., principal gender).
>> The literature suggests that the best approach to this problem is to
>> model these equations jointly and then individually test for
>> differences between the coefficients in X. To complicate matters
>> somewhat, teachers are nested within principals so sureg or mvreg
>> can't be used, since neither can accommodate the clustering. I have
>> pursued several suggestions from colleagues and archived statalist
>> posts (e.g., http://www.stata.com/statalist/archive/2009-04/msg01157.html)
>>  that has landed me a bit further from my comfort zone that I'd like.
>> I'd like to present what I have done thus far and hear if anyone has
>> criticism or alternative suggestions.
>>
>> reg T X
>>     estimates store t1
>> reg P X
>>     estimates store p1
>> suest t1 p1, vce(cluster prinid)
>> foreach x in X {
>>     test _b[t1_mean:`x'] - _b[p1_mean:`x'] = 0
>> }
>>
>> In terms of an interpretation, I'd like to use the t1_mean equation
>> from the suest results to make statements about how each of X factors
>> relate to teachers' perceptions of leadership effectiveness; use
>> p1_mean suest results to make statements about how each of X factors
>> relate to the principals' perceptions of their own leadership
>> effectiveness; and use the test results to make statements about how
>> each of X factors relate to the teacher - principal gap. Kind thanks
>> for your thoughts and insights.
>>
>> Peter
>> peter.t.goff@vanderbilt.edu
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/