Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Interrater agreement: finding the problematic items
From
Nick Cox <[email protected]>
To
"[email protected]" <[email protected]>
Subject
Re: st: Interrater agreement: finding the problematic items
Date
Fri, 14 Jun 2013 16:34:09 +0100
For "Chronbach" read "Cronbach".
Some Statalist members are well versed in psychometrics but I see no
reason why more general statistical ideas should not relevant too. The
standard deviation of ratings for each item would be one measure of
disagreement. Perhaps better ones would be the sum of squared
probabilities or the entropy of the probability distribution for the
rating.
Nick
[email protected]
On 14 June 2013 16:11, Ilian, Henry (ACS) <[email protected]> wrote:
> HI,
>
> I'm doing an interrater agreement study on a case-reading instrument. There are five reviewers using an instrument with 120 items. The ratings scales are ordinal with either two, three or four options. I'm less interested in reviewer tendencies than I am in problematic items, those with high levels of disagreement.
>
> Most of the interrater agreement/interrater reliability statistics look at reviewer tendencies. I can see two ways of getting at agreement on items. The first is to sum all the differences between all possible pairs of reviewers, and those with the highest totals are the ones to examine. The other is Chronbach's alpha. Is there any strong argument for or against either approach, and is there a different approach that would be better than these?
>
> Thanks
>
> Henry
>
> Confidentiality Notice: This e-mail communication, and any attachments, contains confidential and privileged information for the exclusive use of the recipient(s) named above. If you are not an intended recipient, or the employee or agent responsible to deliver it to an intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and delete this communication from your computer. Thank you.
>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/faqs/resources/statalist-faq/
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/