Re: st: Cut-off point for ROC curve using parametric and non-parametric method

["Roger B. Newson" <r.newson@imperial.ac.uk>]

The Youden index can be estimated using -somersd-, for a given binary 
predictor (or binary predictors), by calculating Somers' D of the binary 
predictor (or binary predictors) with respect to the outcome. For 
instance, if the binary (0,1) predictor is -binpred-, and the binary 
(0,1) outcome is -disease-, then you might type:

somersd disease binpred, tdist transf(z)

to get an asymmetric confidence interval for the Youden index. And, if 
you have 2 binary predictors (-binpred1- and -binpred2-), then you might 
type:

somersd disease binpred1 binpred2, tdist
lincom (binpred1-binpred2)/2

to get a symmetric confidence interval for half the difference between 2 
Youden indices. This half-difference can tell us whether -binpred1- or 
-binpred2- is a better predictor of disease.

In your case, you seem to be deriving binary predictors by applying a 
threshold to a continuous predictor (-p-). This is a trivial extension 
to the method illustrated above.

However, in your case, the continuous predictor is calculated (using 
-predict-) from a model fitted to the same data that you later use to 
estimate the Youden index. If you do this, then the Youden index will be 
correct, but the confidence limits may not (because there is no rigorous 
proof that the coverage probability will be correct). If you want to 
compute confidence limits for Youden indices, and their differences, 
which can be taken seriously, then you should probably split the data 
into a training set and a test set, fit the model to the training set, 
compute the continuous predictor using out-of-sample prediction in the 
test set, and estimate the Youden indices and their differences in the 
test set. The issues involved are discussed in Newson (2010).

I hope this helps.

Best wishes

Roger

References

Newson RB. Comparing the predictive power of survival models using 
Harrell’s c or Somers’ D. The Stata Journal 2010; 10(3): 339–358. 
Download pre-publication draft from
http://www.imperial.ac.uk/nhli/r.newson/papers.htm#papers_in_journals

Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: r.newson@imperial.ac.uk
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/

Opinions expressed are those of the author, not of the institution.

On 21/01/2013 13:25, Pham Ngoc Minh wrote:
> Dear Roger,
> Thank you very much for your helpful suggestion. While I am reading
> how to use -somersd- package for estimating Youden index, I found
> -senspec- package convenient to calculate sensitivity (sn) and
> specificity (sp). However, I'm not sure about programs below are
> accurate for estimating age-adjusted sn and sp and their subsequent
> youden index.
>
> logit metabo bmi age
> predict p if e(sample)
> senspec metabo p, sensitivity(sn) specificity(sp)
> gen youdenid= sn-(1-sp)
> egen youdenidmax= max(youdenid)
> gen distance = sqrt((1-sn)^2 + (1-sp)^2)
> egen mindistance = min(distance)
> /* Identify cut-off point */
> list sn sp  youdenidmax   distance bmi if abs(youdenid -youdenidmax)<0.000001
> list sn sp  youdenid     mindistance bmi  if abs(distance -
> mindistance)<0.000001
>
> And evaluating threshold of BMI based on Youden index or shortest
> distance from the curve to upper-left corner.
>
> I'd appreciate your responses to ensure the above commands.
> sincerely,
> Pham Ngoc Minh
>
> On Mon, Jan 21, 2013 at 1:30 AM, Roger B. Newson
> <r.newson@imperial.ac.uk> wrote:
> > If your aim is to estimate the Youden index, then you can use the -somersd-
> > package, which you can download from SSC. The Youden index is an alternative
> > name for Somers' D between 2 binary variables. The -somersd- package can
> > also be used, together with -lincom-, to calculate the difference between 2
> > Youden indices for 2 diagnostic tests for the same disease.
> >
> > I hope this helps.
> >
> > Best wishes
> >
> > Roger
> >
> > Roger B Newson BSc MSc DPhil
> > Lecturer in Medical Statistics
> > Respiratory Epidemiology and Public Health Group
> > National Heart and Lung Institute
> > Imperial College London
> > Royal Brompton Campus
> > Room 33, Emmanuel Kaye Building
> > 1B Manresa Road
> > London SW3 6LR
> > UNITED KINGDOM
> > Tel: +44 (0)20 7352 8121 ext 3381
> > Fax: +44 (0)20 7351 8322
> > Email: r.newson@imperial.ac.uk
> > Web page: http://www.imperial.ac.uk/nhli/r.newson/
> > Departmental Web page:
> > http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/
> >
> > Opinions expressed are those of the author, not of the institution.
> >
> >
> > On 19/01/2013 03:38, Pham Ngoc Minh wrote:
> > > Dear Statalist,
> > >
> > > I want to construct age-adjusted (age is a continuous variable) optimal
> > > cut-off points for body mass index (bmi)  and waist circumference (wc) in
> > > determining metabolic syndrome (metabo) for a large study. My question is
> > > how can I calculate sensitivity and specificity for bmi and wc with
> > > adjustment for age, based on which I can calculate Youden index. For
> > > non-parametric method, I know only about using command: roctab metabo bmi,
> > > detail or senspec metabo bmi, se(varname1) spe(varname2)  (without
> > > covariate adjustment). Moreover, how can I calculate age-adjusted
> > > sensitivity and specificity using parametric ROC analysis. It is because
> > >    using non-parametric method showed P less than 0.01, although AUC for
> > > bmi and wc was somewhat similar.
> > > Thank you for your guidance in advance.
> > >
> > > Pham Ngoc Minh
> > > Thai Nguyen University Faculty of Public Health, Vietnam
> > > *
> > > *   For searches and help try:
> > > *   http://www.stata.com/help.cgi?search
> > > *   http://www.stata.com/support/faqs/resources/statalist-faq/
> > > *   http://www.ats.ucla.edu/stat/stata/
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/faqs/resources/statalist-faq/
> > *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/