Re: st: Evaluating the importance of interaction effects in logistic regression

[Maarten Buis <maartenlbuis@gmail.com>]

On Tue, Aug 30, 2011 at 11:31 PM, Thomas Speidel wrote:
> Maarten L. Buis (2010) "Stata tip 87: Interpretation of interactions in
> non-linear models", The Stata Journal, 10(2), pp. 305-308.)
>
> writes in the context of his first example that the interaction between the
> two dischotomies black and collgrad "is not significant" (the reported
> p-value is in fact 0.161).  However, another reference Maarten cited:
>
> Edward Norton, Hua Wang, and Chunrong Ai (2004) "Computing interaction
> effects and standard errors in logit and probit models" The Stata Journal,
> 4(2): 154-167.
>
> says that "The statistical significance cannot be determined from the
> z-statistic reported in the regression output" (p.1).
> I am now confused on the appropriate way of identifying significant
> interactions.  This sentence has confused me.  Regardless of interpretation,
> how does one assess the importance of an interaction?

You cannot determine the significance without deciding how you want to
interpret the results. Interpretation and significance are not
independent, as the first determines the null hypothesis of the
second. This is the key bit of information you need in order to see
that I (Maarten) and Norton et al. do not disagree, even though these
quotes make it seem like we do. I showed in my Stata tip how to
interpret interaction terms as ratios of odds ratios, in which case
you can interpret the p-values as the test that this ratio equals 1,
i.e. there is no difference between black and white females in the
effect (measured as odds ratios) of collgrad. Norton et al. want to
interpret effects as marginal effects, i.e. as differences in
probabilities rather than ratios of odds, and the marginal effect as
differences (rather than ratios) in marginal effects. As a
consequence I and Norton et al. want to test different hypotheses, and
obviously get different results.

> For example:
> ********************************************************************************
> sysuse nlsw88, clear
> gen byte high_occ = occupation < 3 if occupation < .
> drop if race==3
> logistic high_occ race##collgrad , nolog
<snip>
> is it correct to say that the interaction race * collgrad is not important
> because its p-value is 0.161?

Such a conclusion is always wrong as a significance test cannot
determine whether or not a variable is "important". This may sound
pedantic, but this misunderstanding is probably the root cause of your
confusion. As soon as you regard tests as testing a specific
null-hypothesis the distinction between me and Norton et al. becomes
much easier to understand. As I stated above this tests the hypothesis
that the ratio of the effect (measured in odds ratios) of collgrad for
white women and the effect of collgrad for black women equals 1, i.e.
that these effects are equal. If that is a hypothesis that is of
interest to you, than this test is fine. If this hypothesis is not of
interest to you, than this test is not fine.

> What if, for example, we had 3 levels to race:
>
> ********************************************************************************
> sort idcode
> replace race = 3 in 1/300
> logistic high_occ race##collgrad, nolog
<snip>
> and we want to evaluate the overall importance of the interaction between
> race and collgrad (i.e. jointly)?  Is it approriate to use the likelihood
> ratio test to compare the model without interaction to the model with
> interaction, and determine the importance of the interaction effect
> according based on the results of LR test?

That depends on whether you want to interpret your results in terms of
odds ratios and your interaction terms as ratios of odds ratios. If
you want to do that, than what you propose is one way of doing that.

Hope this helps,
Maarten

--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany


http://www.maartenbuis.nl
--------------------------

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/