Kieran McCaul posted results from a randomized parallel-group design study to
illustrate the use of conditional logistic regression. The study randomized
households to an intervention designed to promote banning of smoking in the
home. Policy in the home was measured before and after intervention. Kieran
invited Ricardo and I to respond with what we think of advocating conditional
logistic regression to assess the efficacy of the intervention for before-and-
after studies based upon the results posted for that study.
I don't claim to speak for Ricardo, but his original question related to
imbalances in the baseline rates of the outcome between the two parallel
intervention groups. It appears that Kieran's study was successful in its
randomization (or used stratified randomization and didn't lose too many
households to dropout), because the proportions of households banning smoking
at baseline were nearly identical between the intervention groups. With
essentially identical rates of baseline, there would be little or no cause for
concern about confounding due to it and little statistical difference in
including baseline as a covariate. And, in fact, both conditional logistic
regression approach and the so-called ANCOVA-like multiple logistic regression
approach give essentially similar results in this balanced study. (I think the
same would have obtained for Ricardo's study had the baseline rates of seatbelt
use been similar between the two intervention groups.)
But, let's look at the issue of which approach is more suitable when the
concern is, as it was for Ricardo, to analyze an intervention effect _in the
face of an imbalance in the baseline rates of an outcome_.
If Kieran will indulge me one more time to use a fictional dataset to
illustrate a point, let's say that Kieran's randomization method did not
stratify on baseline household smoking policy, and suffered an unfortunate
imbalance due to chance, for instance a 50 : 50 ratio of households banning
smoking at baseline in the nonintervention group, but a 75 : 25 ratio in the
intervention group. Let's say that 2 of the 50 households that previously
banned smoking in the nonintervention group now permit it, a worsening of 4%
(if your health policy is to ban smoking), and that only 1 of the 50 households
that didn't ban smoking now do so in the nonintervention group, a meager
improvement of 2%. Let's say that 4 of the 75 households that banned smoking
at baseline switched and permitted smoking in the home after the intervention,
and 2 of the 25 households that didn't ban smoking switched as a result of the
intervention. The results of the intervention are a slightly greater 5.3%
worsening (compare to 4%) in the former nonbanning household population, but a
much greater 8% (compare to 2%) improvement among the formerly permissive
households.
Now, the effects of intervention are no great shakes, but I think that it would
be safe to say that it's not *nothing*, especially if you somehow take into
account the possible confounding effect of the chance unfortunate imbalance in
baseline policy between treatment groups.
But, by the conditional logistic regression approach, it *is* nothing--the odds
ratio for both nonintervention and intervention groups is 0.5 (McNemar's test
uses only the off-diagonal values and ignores the diagonal values) so the ratio
of the two odds ratios is 1.0, and this is what the conditional logistic
regression dutifully reports: the period term is 0.5 and the interaction
term's odds ratio is 1.0 with a Z-statistic of 0.00 and a p-value of 1.00.
Granted, the confidence interval encompasses a lot, but the point estimate and
hypothesis test for the interaction term (which is ostensibly the effect of
intervention) just don't give the same take-home message as inspection of the
data. So, my conclusion differs from Kieran's on this; I don't think that
conditional logistic regression is valid to test for differences between
treatment effects (differences between treatment differences, which are between-
subject effects) in parallel-group designs with a repeated binary outcome
measure, especially in the presence of baseline differences in the outcome
measure, which are ignored in the conditional logistic model.
In contrast, the ANCOVA-like, baseline-as-covariate multiple regression
approach does provide a separate, and I think competent, handling of baseline
differences and their potential for confounding. In the fictitious example,
this approach shows the pronounced effect of baseline smoking policy as
expected, and it shows that the odds ratio for intervention isn't 1.0 given
baseline differences between intervention groups. The saturated model (with
the interaction term) also helps to put the potential for confounding into
perspective. (The do-file for all of this is below for anyone interested.)
It seems that at least some of the discrepancy between the two approaches
reflects Simpson's paradox. This is the same underlying phenomenon that
results in bias in logistic regression coefficients (and in nonlinear
regression, in general) when important covariates are left out of the model.
This is what Frank E. Harrell Jr.'s lecture dealt with in the URL given in my
last posting. And it relates to the "noncollapsibility of odds ratios" that
epidemiologists sometimes refer to.
In fairness to us all (Kieran, Ricardo and me), it seems that the matter of
which approach is better isn't completely settled even for *linear* models,
where this incollapsibility-of-odds-ratios phenomenon and the incidental
parameters problem don't apply: there is a thread ("Repeated measures and
including time zero response as baseline covariate") on sci.stat.consult that
was started on May 7 of last year by Frank Harrell. Professor Harrell wrote a
well received book on regression modeling and is now chairman of a department
of biostatistics, yet even he asks, "Has anyone come across some practical
guidance for when to include the first measured response (at time zero) as a
baseline covariate as opposed to the first repeated measurement in a
longitudinal data analysis?"
Joseph Coveney
-------------------------------------------------------------------------------
clear
tempfile tmp
set obs 100
generate byte ban0 = _n > _N / 4
generate byte ban1 = ban0
replace ban1 = !ban1 in 50/53
replace ban1 = !ban1 in 1/2
*
* Intervention group
*
display 4 / 75 // switching by banners
display 2 / 25 // switching by permitters
mcc ban1 ban0
generate byte intervention = 1
save `tmp'
clear
set obs 100
generate byte ban0 = _n > _N / 2
generate byte ban1 = ban0
replace ban1 = !ban1 in 50/52
*
* Nonintervention group
*
display 2/50 // switching by banners
display 1/50 // switching by permitters
mcc ban1 ban0
generate byte intervention = 0
append using `tmp'
erase `tmp'
generate byte iac = ban0 * intervention
generate int id = _n
logistic ban1 ban0 intervention iac, or nolog
estimates store A
logistic ban1 ban0 intervention, or nolog
estimates store B
lrtest A B
logistic ban1 ban0, or nolog
lrtest A .
lrtest B .
quietly {
reshape long ban, i(id) j(period)
replace iac = period * intervention
}
clogit ban period intervention iac, group(id) or nolog
xtgee ban period intervention iac, i(id) family(binomial) link(logit) ///
corr(exchangeable) nmp eform nolog
exit
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
