To evaluate students on clinical rotations, our medical school (like
many, I suppose) uses a form on which the evaluator assigns an integer
score for each of 10 competencies. The scale runs from 1 to 5; 1 is
worst, 5 is best. Generally, each student is evaluated by 10-11
different people, yielding 10-11 observations on each competency for
each student.
At least one of the clerkship directors is in the habit of calculating a
student's mean score on each competency. To two decimal places. The
competency means are then averaged again, using the mean, to produce a
single summative number for each student. To 3 decimal places. This
number then counts a certain percentage toward the final clerkship grade
(along with written exams and other exercises.)
I may be wrong, but I have some reservations about calculating means of
scores on a 5-point Likert scale. To me it seems like an ordinal scale.
I keep making the claim that the proper measure of central tendency in
this case would be the median. None of my colleagues agree; I get the
feeling they don't see any problem with putting arbitrary consecutive
integer values on the different levels of performance. My point is
that, while it is neat and tidy and perhaps intuitive, we have no
evidence that the labeled levels on the performance scale are equally
"spaced."
My concern is with the end result: in the final accounting, students
are sorted by final score and end up being separated by fractions of a
point. I would consider these students to be of essentially equal
abilities, for all practical purposes. Yet sometimes the cut-points for
our categories of final grade (Honors, High Pass, Pass, Conditional,
Fail) end up falling between two very closely spaced students.
What are your collective thoughts on this?
Thanks.
--Chris
--
Christopher W. Ryan, MD
SUNY Upstate Medical University Clinical Campus at Binghamton
and Wilson Family Practice Residency, Johnson City, NY
cryanatbinghamtondotedu
GnuPG and PGP public keys available at http://pgp.mit.edu
"If you want to build a ship, don't drum up the men to gather wood,
divide the work and give orders. Instead, teach them to yearn for the
vast and endless sea." [Antoine de St. Exupery]
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
