st: Re: Kappa PABAK

[Red Owl <rh.redowl@liu.edu>]

I am not aware of a user-written Stata program for calculating PABAK,
but it is a straightforward calculation.

For a discussion of PABAK (and some cautions and alternatives) see:
Cunningham, M. (2009). More than just the kappa coefficient: A program 
   to fully characterize inter-rate reliability between two raters. 
   Paper 242-2009. SAS Global Forum 2009. Retrieved from 
   http://support.sas.com/resources/papers/proceedings09/242-2009.pdf

In a 2 x 2 table, the proportion of observed agreement is calculated as:
p(observed) = (a + d) / N.
and, then:
PABAK = 2 * p(observed) - 1

For polytomous ratings, you could also consider linear or quadratically 
weighted kappa, as provided in -kap- and -kapwgt- in Stata and then
report the prevalence and bias indices shown below along with kappas. 

In  the 2 x 2 case, you can use the following simple formulas from 
p. 2 in Cunningham(2009) to calculate prevalence and bias indices.
(These formulas can easily be extended.)
Prevalence Index = (a-d)/N
Bias Index = (b-c)/N

Below is a simple toy example in Stata for a 2 x 2 case:

* Build rater x rater data matrix from Figure 2, Cunningham (2009).
matrix input A = (95, 4 \ 1, 0)
* Calculate and display PABAK plus prevalence and bias indices.
scalar pabak = 2 * (A[1,1]+A[2,2])/(A[1,1]+A[1,2]+A[2,1]+A[2,2])-1
scalar prevndx = (A[1,1]-A[2,2])/(A[1,1]+A[1,2]+A[2,1]+A[2,2])
scalar biasndx = (A[1,2]-A[2,1])/(A[1,1]+A[1,2]+A[2,1]+A[2,2])
di %5.4f "PABAK = " pabak
di %5.4f "Prevalence Index = " prevndx
di %5.4f "Bias Index = " biasndx


Red Owl
redowl@liu.edu


>> Is there a user command for prevalence adjusted bias adjusted kappa
>> (PABAK)? I have searched the statalist and can't seem to find one.
>> 
>> Thank you,
>> 
>> Jordan
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/