On Tue, Rena Jenkins asked:
> I would like to calculate an intraclass correlation coefficient, to
> measure agreement between raters. I know that you can use the
> command "loneway" along these lines:
>
> loneway measurement rater
>
> ...but I have measurements by the same group of raters on several
> different objects (the objects being indentified by the variable "ID").
> I want to use the two-way model:
>
> anova measurement id rater
>
> How would I calculate an ICC from the sums of squares given in the
> resulting ANOVA?
>
Instead of going to the two random factor case right away, let's make
sure we know how to compute the intraclass correlation for one random
factor.
The formulas for this can be found in Kuehl's book on "Design of
Experiments" (page 155). The example I am using here is from the manual
[R] loneway (page 339, version 8). The approach is to first count the
number of repetitions for each level of factor A, then compute the
variance components, and lastly compute ICC.
*----one random factor--------------*
clear
webuse auto7
keep if nummake==4
*----------------
local A="manufacturer_grp"
local depvar="mpg"
loneway `depvar' `A'
table `A'
local N=r(N)
egen tag=tag(`A')
egen sum=sum(tag)
local n=`N'/sum[1]
display `n'
drop sum tag
anova `depvar' `A'
local MSA=e(mss)/e(df_m)
display `MSA'
local MSE=e(rss)/e(df_r)
display `MSE'
local sigA=(`MSA'-`MSE')/`n'
local ICC=(`sigA')/(`sigA'+`MSE')
display `ICC'
*---------------------
You may notice I am assuming a balanced design. If the design is not
balanced, then there are slightly more complicated formulas for the
variance components (also given in Kuehl).
OK, now for two random factors. For these formulas I must turn to the
comprehensive experimental design text by Winer et. al. "Statistical
Principles of Experimental Design" (page 413). The formula is what you
would expect it to be: ICC=variance of A/(total variance). So again, we
count the number of repetitions, and this time the number of levels of
each factor, compute the variance components, and then ICC.
This time I am going to generate my own balanced dataset. Again, if your
design is not balanced, you just have to work harder to get the variance
components (the formulas for this are in Winer).
*----two random factors--------------*
*--- generate balanced data
clear
set obs 36
generate A=int(_n/6+1)
replace A=1 if A==7
table A
bysort A: generate B=int(_n/2+1)
replace B=1 if B==4
generate response=uniform()
*----------------
local depvar="response"
local A="A"
local B="B"
table `A' `B'
local N=r(N)
display `N'
egen tag=tag(`A')
egen sum=sum(tag)
local p= sum[1]
display `p'
drop sum tag
egen tag=tag(`B')
egen sum=sum(tag)
local q= sum[1]
display `q'
drop sum tag
local n=`N'/(`p'*`q')
display `n'
anova `depvar' `A' `B' `A'*`B'
test `A'
local MSA=r(ss)/r(df)
display `MSA'
local MSE=r(rss)/r(df_r)
display `MSE'
test `B'
local MSB=r(ss)/r(df)
display `MSB'
test `A'*`B'
local MSAB=r(ss)/r(df)
display `MSAB'
local sigA=(`MSA'-`MSAB')/(`n'*`q')
local sigB=(`MSB'-`MSAB')/(`n'*`p')
local sigAB=(`MSAB'-`MSE')/`n'
local ICC=(`sigA')/(`sigA'+`sigB'+`sigAB'+`MSE')
display `ICC'
*---------------------
Notice, we have to work a bit harder to get the mean squares for each of
the factors this time, since they are not in the saved results after
-anova-. But, that is where -test- comes in handy, because they are in
the results after -test-.
By the way, both the books I have refered to above are available at the
Stata bookstore:
http://www.stata.com/bookstore/doe.html
http://www.stata.com/bookstore/sped.html
yours,
May
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
