Re: st: Cut-off point for ROC curve using parametric and non-parametric method

[Pham Ngoc Minh <phamngocm@gmail.com>]

Dear Roger,

Great thanks to your intuitive guide and detailed explanations about
'somersd' package for this purpose. I'll reach the approach to
estimating CI for my case with reference to your paper in Stata
journal.

Thank you very much
sincerely,
Pham Ngoc Minh

On Tue, Jan 22, 2013 at 1:02 AM, Roger B. Newson
<r.newson@imperial.ac.uk> wrote:
> The Youden index can be estimated using -somersd-, for a given binary
> predictor (or binary predictors), by calculating Somers' D of the binary
> predictor (or binary predictors) with respect to the outcome. For instance,
> if the binary (0,1) predictor is -binpred-, and the binary (0,1) outcome is
> -disease-, then you might type:
>
> somersd disease binpred, tdist transf(z)
>
> to get an asymmetric confidence interval for the Youden index. And, if you
> have 2 binary predictors (-binpred1- and -binpred2-), then you might type:
>
> somersd disease binpred1 binpred2, tdist
> lincom (binpred1-binpred2)/2
>
> to get a symmetric confidence interval for half the difference between 2
> Youden indices. This half-difference can tell us whether -binpred1- or
> -binpred2- is a better predictor of disease.
>
> In your case, you seem to be deriving binary predictors by applying a
> threshold to a continuous predictor (-p-). This is a trivial extension to
> the method illustrated above.
>
> However, in your case, the continuous predictor is calculated (using
> -predict-) from a model fitted to the same data that you later use to
> estimate the Youden index. If you do this, then the Youden index will be
> correct, but the confidence limits may not (because there is no rigorous
> proof that the coverage probability will be correct). If you want to compute
> confidence limits for Youden indices, and their differences, which can be
> taken seriously, then you should probably split the data into a training set
> and a test set, fit the model to the training set, compute the continuous
> predictor using out-of-sample prediction in the test set, and estimate the
> Youden indices and their differences in the test set. The issues involved
> are discussed in Newson (2010).
>
>
> I hope this helps.
>
> Best wishes
>
> Roger
>
> References
>
> Newson RB. Comparing the predictive power of survival models using Harrell’s
> c or Somers’ D. The Stata Journal 2010; 10(3): 339–358. Download
> pre-publication draft from
> http://www.imperial.ac.uk/nhli/r.newson/papers.htm#papers_in_journals
>
>
> Roger B Newson BSc MSc DPhil
> Lecturer in Medical Statistics
> Respiratory Epidemiology and Public Health Group
> National Heart and Lung Institute
> Imperial College London
> Royal Brompton Campus
> Room 33, Emmanuel Kaye Building
> 1B Manresa Road
> London SW3 6LR
> UNITED KINGDOM
> Tel: +44 (0)20 7352 8121 ext 3381
> Fax: +44 (0)20 7351 8322
> Email: r.newson@imperial.ac.uk
> Web page: http://www.imperial.ac.uk/nhli/r.newson/
> Departmental Web page:
> http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/
>
> Opinions expressed are those of the author, not of the institution.
>
> On 21/01/2013 13:25, Pham Ngoc Minh wrote:
>>
>> Dear Roger,
>> Thank you very much for your helpful suggestion. While I am reading
>> how to use -somersd- package for estimating Youden index, I found
>> -senspec- package convenient to calculate sensitivity (sn) and
>> specificity (sp). However, I'm not sure about programs below are
>> accurate for estimating age-adjusted sn and sp and their subsequent
>> youden index.
>>
>> logit metabo bmi age
>> predict p if e(sample)
>> senspec metabo p, sensitivity(sn) specificity(sp)
>> gen youdenid= sn-(1-sp)
>> egen youdenidmax= max(youdenid)
>> gen distance = sqrt((1-sn)^2 + (1-sp)^2)
>> egen mindistance = min(distance)
>> /* Identify cut-off point */
>> list sn sp  youdenidmax   distance bmi if abs(youdenid
>> -youdenidmax)<0.000001
>> list sn sp  youdenid     mindistance bmi  if abs(distance -
>> mindistance)<0.000001
>>
>> And evaluating threshold of BMI based on Youden index or shortest
>> distance from the curve to upper-left corner.
>>
>> I'd appreciate your responses to ensure the above commands.
>> sincerely,
>> Pham Ngoc Minh
>>
>> On Mon, Jan 21, 2013 at 1:30 AM, Roger B. Newson
>> <r.newson@imperial.ac.uk> wrote:
>>>
>>> If your aim is to estimate the Youden index, then you can use the
>>> -somersd-
>>> package, which you can download from SSC. The Youden index is an
>>> alternative
>>> name for Somers' D between 2 binary variables. The -somersd- package can
>>> also be used, together with -lincom-, to calculate the difference between
>>> 2
>>> Youden indices for 2 diagnostic tests for the same disease.
>>>
>>> I hope this helps.
>>>
>>> Best wishes
>>>
>>> Roger
>>>
>>> Roger B Newson BSc MSc DPhil
>>> Lecturer in Medical Statistics
>>> Respiratory Epidemiology and Public Health Group
>>> National Heart and Lung Institute
>>> Imperial College London
>>> Royal Brompton Campus
>>> Room 33, Emmanuel Kaye Building
>>> 1B Manresa Road
>>> London SW3 6LR
>>> UNITED KINGDOM
>>> Tel: +44 (0)20 7352 8121 ext 3381
>>> Fax: +44 (0)20 7351 8322
>>> Email: r.newson@imperial.ac.uk
>>> Web page: http://www.imperial.ac.uk/nhli/r.newson/
>>> Departmental Web page:
>>>
>>> http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/
>>>
>>> Opinions expressed are those of the author, not of the institution.
>>>
>>>
>>> On 19/01/2013 03:38, Pham Ngoc Minh wrote:
>>>>
>>>>
>>>> Dear Statalist,
>>>>
>>>> I want to construct age-adjusted (age is a continuous variable) optimal
>>>> cut-off points for body mass index (bmi)  and waist circumference (wc)
>>>> in
>>>> determining metabolic syndrome (metabo) for a large study. My question
>>>> is
>>>> how can I calculate sensitivity and specificity for bmi and wc with
>>>> adjustment for age, based on which I can calculate Youden index. For
>>>> non-parametric method, I know only about using command: roctab metabo
>>>> bmi,
>>>> detail or senspec metabo bmi, se(varname1) spe(varname2)  (without
>>>> covariate adjustment). Moreover, how can I calculate age-adjusted
>>>> sensitivity and specificity using parametric ROC analysis. It is because
>>>>    using non-parametric method showed P less than 0.01, although AUC for
>>>> bmi and wc was somewhat similar.
>>>> Thank you for your guidance in advance.
>>>>
>>>> Pham Ngoc Minh
>>>> Thai Nguyen University Faculty of Public Health, Vietnam
>>>> *
>>>> *   For searches and help try:
>>>> *   http://www.stata.com/help.cgi?search
>>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>>> *   http://www.ats.ucla.edu/stat/stata/
>>>>
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>> *   http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/