Re: Re: st: Re: cutoff point for ROC curve

[Clyde Schechter <clyde.schechter@gmail.com>]

I would advise Michael Stewart not to seek some arbitrary formula for
the optimal cut-off point.  He doesn't say what is being classified,
but regardless, the substantive issue is the trade-off between two
types of misclassification errors: false negatives and false
positives.  Both types of error have consequences, usually different.
To find an optimal cut-point requires assigning a loss to each type of
error and then expressing the expected loss in terms of sensitivity,
specificity and prevalence of the attribute being identified by the
classification.  Then you pick the cut-off which minimizes the
expected loss.

My practical experience with this process is that people are often
reluctant to quantify the losses associated with each type of error,
because the losses are often of a qualitatively different nature.  For
example, a missed diagnosis may lead to loss of life, whereas a false
positive diagnosis may lead to unnecessary surgery.  How does one
assign values to those?  Not easily.

So it feels more comfortable to seize on some simple formula, such as
the sum of sensitivity and specificity.  Nevertheless, if you don't
really quantify and compare the losses associated with each type of
error, applying some arbitrary formula will give you only the
illusion, not the reality, of optimality.  One is simply optimizing an
arbitrary quantity that bears no relation to the matter at hand.

Clyde Schechter
Dept. of Family & Social Medicine
Albert Einstein College of Medicine
Bronx, New York, USA
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/