st: Advice on xtmixed specification,pre/post two-group design

[Brandon Olszewski <olszewski.brandon@gmail.com>]

Hi statalist:
I’m hoping for advice on modeling clustered, two-condition, pre/post
scores on assessment. I have used -xtmixed- a number of different
ways, but am unsure about how to best model my data. I do have and use
Rabe-Hesketh & Skrondal's 2nd edition for multilevel modeling using
stata, so references to that are very helpful for me. I’ve also looked
on the multilevel list at JISCMail, and as a result, only feel more
unsure about one strategy over another. Data are in –long- format,
with two observations (time 1 and 2) reported for every teacher-nested
student. Here’s a simplified version of what I’ve got:
>score = Student scores on an assessment at two times (time 1 and time 2)
>control = control or treatment group
>id = unique student ID
>tch_id = teacher ID
>female = female or not
>urm = under-represented minority
>english = if English is the student’s native language
Option 1: I want to know whether or not scores increase over time, so
intuitively, it seems like I would want to set it up like this:
>xtmixed score time female urm english control time#control || tch_id: || id: , mle
…or if I want to let slopes for group assignment (control) vary…
>xtmixed avg_AR time female urm english control time#control || tch_id: control, cov(unstruct) || id: , mle
Option 2: But from what I can tell, having only two time points might
merit a different model. Instead, I could regress the post score
(score_t2) on the pre score (score_t1) and other inde vars, as in:
>xtmixed score_t2 female urm english score_t1control time#control || tch_id: control, cov(unstruct) || id: , mle
Option 3: I could regress change in scores on the inde vars. There
seems to be a rich history of debate around whether or not change
outcomes (score_change) are reliable, and I’m definitely not
experienced enough to confidently sift through that. It seems like
that would be modeled as follows:
>xtmixed score_change female urm english control time#control || tch_id: control, cov(unstruct) || id: , mle
…Results…
Box plots of the sample (of ~1,900 students) show a clear pattern that
regression should pick up…a positive coefficient for the treatment
group, and probably a positive interaction between treatment and time
(the treatment group exhibits a steeper rate of growth). I ran each of
the three models I mentioned with my data. Option 1 provided the
results that made the most sense and greatest number of significant
coefficients, including group, urm, time, and time*group. Option 2
yielded similar results. Option 3 results read totally differently –
significance washed out, and coefficient signs changed.
Any advice on how to proceed, and rationales for doing so would be
greatly appreciated. Thanks.
Brandon Olszewski

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/