st: Re: Kappa for multiple raters and paired body parts

["Joseph Coveney" <jcoveney@bigplanet.com>]

Garry Anderson wrote:

I wish to estimate a single kappa (SE or 95%CI) when there are 4 raters
that each rate the left and right eyes of about 150 patients. The
response for each eye is binary. Estimation of kappa (SE) can be done
separately for the left eye and the right eye using -kap- or -kapci-,
however I am unsure as to how to include both eyes and take account of
the non-independence of eyes. Schouten (1993) describes the methodology
for two raters.

Schouten HJA (1993) Estimating kappa from binocular data and comparing
marginal probabilities. Statistics in Medicine 12: 2207-2217.

Any suggestions would be appreciated.

--------------------------------------------------------------------------------

Well, kappa for binary scores is an intraclass correlation coefficient (ICC).  
How about using -xtmelogit- to fit a cross-classified random-effects model to 
the data with i.side (right or left eye) as a fixed effect, and then use the 
patients' and raters' variance components, along with the logistically 
distributed residual (pi^2 / 3), to compute the ICC (patients' divided by the 
sum of patients', raters' and residual)?  You can get the (transformed) 
variance components from the parameter vector, e(b).  I'm guessing that 
bootstrapping is the best bet for the confidence interval.  But -nlcom- is 
also worth looking into for this.

Joseph Coveney


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/