st: Significance of between-cluster variance in xtlogit

[Michael Josten <mj.josten@googlemail.com>]

Hi there,

I am currently dealing with multilevel logistic regression models to
investigate interviewer and interview mode effects. I mainly use
xtlogit for this and everything is working fine except for the
interpretation of the actual significance of the observed
between-cluster variance caused by the interviewers. Throughout all of
my models, from the random intercept only model to the full random
intercept model with all respondent and interviewer variables, rho
continuously is considerably high (0.3-0.2) which is indicative of a
remarkable interviewer effect. The provided likelihood ratio test for
the null hypothesis that the residual between-cluster variance is zero
indicates that rho indeed is highly statistically significant for each
single model. For me this is a sign that the multilevel design is
appropriate because otherwise I would get biased estimators.

However, a second result is contradictory to this interpretation: the
log cluster variance - indicated as “lnsig2u” - doesn’t seem to be
significant anymore at the moment I control for the interview mode
CATI or CAPI. I can see that looking at the confidence interval which
embraces zero, but I can’t comprehend what kind of statistical test is
performed here even though there is some standard error provided.
Stata manual doesn’t give information about this procedure. However,
this would be a sign that it isn’t the interviewers in general who are
cheating but it is only the difference between CATI and
CAPI-Interviewer.

Even though this makes sense as there are different incentives in the
two modes for interviewers to cheat, there are still two contradictory
findings: on the one hand rho as the proportion of between-variance of
the total variance indicates that there is a significant
between-cluster variance, on the other hand in the same model some
sort of untransparent test comes to the result that exactly this
between-cluster variance isn’t significant for its own. Does anyone
have experience with this kind of problem and can help me as I am not
quite sure on which result to rely on for my interpretation as their
messages are totally different. Thanks!

Best,
Michael

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/