Phil Schumm wrote:
Focusing for a moment on just one outcome, your options are
essentially (1) an analysis of the change scores, or (2) regression
of the outcome on the treatment indicator and the baseline value.
Option (2) can give inconsistent estimates of the treatment effect
due to measurement error in y_1 unless your treatment assignment is
randomized (e.g., Allison 1990), which you indicated is the case
here. The advantage of (2) is that it is more efficient (as you
point out), and yields a result that is often of direct interest
(i.e., the difference between treatment groups in the mean value of
y_2 for a given *observed* value of y_1). There was a thread in the
American Statistician on these issues a while back; Laird (1983) is a
good entry point.
Once you have a model for each outcome that you feel comfortable
with, you can then think about estimating them jointly either to
increase efficiency and/or to permit joint tests (e.g., to construct
a single test of treatment effect for both outcomes). Certainly -
sureg- provides one reasonable approach for doing this. Your other
approach -- multivariate regression in which both outcomes are
regressed on both sets of baseline values -- strikes me as
unjustifiable, unless you are really interested in the effects of the
baseline value of one measure on the post-treatment value of the
other. Of course, if you are really interested in this, then you
also need to consider the effect that measurement error in the
baseline values may have on your analysis.
Personally, whenever I've been faced with an analysis of pre/post
data, I've always started by considering several specific models for
the measurement process and for the effect(s) of the treatment (e.g.,
homogeneous versus heterogeneous, dependent on the baseline value of
the outcome, etc.), and tried to figure out what the implications of
these were for different analyses. There's a limit to what you can
do in terms of estimation with only a single pre and post
measurement, of course, but I have still found this exercise to be
helpful. The papers cited here (and their references) provide
several good examples of what I am talking about.
--------------------------------------------------------------------------------
Thanks again, Phil, for the follow-up. I'll see if I can get my hands on a
copy of the Allison paper. You make several good points.
The study will be conducted in a so-called government-regulated environment.
The analysis must be specified in detail in writing in advance, and so there
isn't much opportunity after the data are in hand to find a model to be
comfortable with.
In accordance with the study's primary objective, the primary analysis is
geared toward hypothesis testing instead of estimation. The use of the
response variable at baseline as a covariate is solely utilitarian, to
increase efficiency. (Of course, should the treatment null hypothesis be
rejected, a natural question arises as to the nature of the effect, and
answering that will involve estimation.)
I'm with you in having reservations about including both responses as
covariates in a multivariate regression. But it might help thinking about
it not as involving a regression left_after upon right_before and vice
versa, but rather as regressing a response vector upon a vector of
predictors--the latter happens to include baseline responses among the other
predictors shared by each element of the response vector. I still
like -sureg-, though. But client's love to see that "exact" that you'll
get in the -manova- printout, and don't like any fiddling with the
denominator degrees of freedom after -sureg-.
I know that when there is randomized assignment to treatment group the
expectation is zero for the correlation between the baseline covariate and
treatment. But I was concerned that a finite-sample realization as a given
study will have a nonzero correlation, in practice. Simulations with
univariate regressions have been reassuring, though, in that there hasn't
been any discernable association between realized correlation coefficient
and regression-coefficient attenuation in sample sizes of 50 per treatment
group. It seems that a pretest response as a covariate doesn't behave like
an errors-in-variables predictor, that, because the source of errors for the
response is the same before and after, the errors can be assigned entirely
to the posttest response.
Joseph Coveney
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
