Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: reliability with -icc- and -estat icc-
From
"JVerkuilen (Gmail)" <[email protected]>
To
[email protected]
Subject
Re: st: reliability with -icc- and -estat icc-
Date
Wed, 27 Feb 2013 10:33:53 -0500
On Wed, Feb 27, 2013 at 10:06 AM, <[email protected]> wrote:
>
> When in doubt, try going back to a reference source (
> www.hongik.edu/~ym480/Shrout-Fleiss-ICC.pdf ) and manually computing
> the ICC. According to the source, “ICC is the correlation between
> one measurement . . . on a target and another measurement obtained on
> that target.” In your case, targets are smartphone software.
Correct.
> By the way, Rater #4 is providing valuable information about rater
> reliability, and so I recommend against eliminating her scores from
> the ICC computation.
I think my interpretation is that Rater 4 is an outlier. The question
is what does that outlier status mean? It's very clear that she's
driving most of the ICC estimate so the notion of a good ICC estimate
from these data themselves is suspect. This is a useful study outcome.
> My take on all that would be that your volunteers need better
> training on evaluating smartphone software in the manner that you
> want it done. Perhaps you and your colleagues could provide more
> explicit instructions on what you’re are looking for in measuring the
> characteristic(s) of the software that you’re trying to measure.
Absolutely true, and is probably the best takeaway of this exercise.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/