Thanks for the response David. I think I should have
clarified a few points:
1. Teeth are from animals not humans and I think that
they are from separate animals. I am not familiar with
his experiment, we only had a short conversation about
it. I understand what you mean about correlated teeth
within animal.
2. There is no gold standard method for these
measurements
3. There is no specific focus on raters' agreement. I
think he wants to show that the method that he used is
accurate enough and gives consistent measurements even
when used by different researchers, or in different
time occasions or finally with varying positioning of
the tooth
4. I think he would also be interrested in estimating
the variabilty induced by the rater, position or time
In fact fiting sequentially models from null (no
covariates) to saturated (all covariates and
interactions) the estimated Var(b_i) component remains
almost constant whereas the estimated Var(e_ij)
slightly decreases (b_i is the random intercept and
e_ij the level-1 residual)
I have to admit that I am not familiar with
reliability analysis thus maybe I am missing something
obvious. Any pointers to relevant methodological
papers, books, or worked examples would be much
appreciated.
Thanks again
Nikos Pantazis
Biostatistician
--- David Airey <[email protected]> wrote:
> > Hi Statalisters,
> >
> > a Dentistry PhD student did some measurements on
> 12
> > teeth with varying conditions and he asked me how
> > could he show that the device used for the
> > measurements is reliable. More specifically each
> one
> > of the 12 teeth has been measured by this device
> by 2
> > raters (a and b) X 2 time points (week 1 and week
> 2) X
> > 6 relative positions = 24 measurements. The goal
> is
> > to show that the discrepancies among measurements
> are
> > not statistically significant.
>
> That is interesting.
>
> I'm not sure whether the student is interested in
> reliability (variance
> component estimation?) or the student wishes to
> compare performance of
> a new method to an accepted method (mean comparisons
> with certain
> assumptions about the variance between methods?).
>
> If the latter, maybe you want a planned one-sided
> comparison against a
> "gold standard". You are only interested in whether
> your new method
> mean is higher (or lower) than an accepted method's
> mean performance.
> You might look at Hsu (1996) Multiple Comparisons:
> Theory and methods
> (Chapman and Hall).
>
> If the former, don't your hypotheses center around
> the rater means and
> or variances? There is a lot of literature on rater
> reliability. Seems
> like the design has a lot of information within two
> raters, but very
> little between rater information from the population
> of possible
> raters.
>
> In your model, level of variation to think about is
> the person. Is each
> tooth from a separate individual? Teeth are
> correlated with person if
> not.
>
>
> >
> > My first thought was to use a 2-level model
> > (measurements nested within tooth) and test the
> rater,
> > time, position effects and maybe their
> interractions.
> > Something like
> >
> > xi:xtreg length i.rater i.time i.position,i(tooth)
> >
> > assuming that a random intercept structure is
> adequate
> > for this experiment.
> > The problem is that this is the first time I deal
> with
> > a problem where the goal is to "prove" that some
> > factors have no significant effects, thus I am not
> > very confident with the aforementioned method.
> >
> > Any thoughts?
> >
> > Nikos Pantazis
> >
> > Biostatistician
>
> *
> * For searches and help try:
> *
> http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
__________________________________
Do you Yahoo!?
Read only the mail you want - Yahoo! Mail SpamGuard.
http://promotions.yahoo.com/new_mail
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
