I too would prefer not to assume Normality and/or homoskedasticity. And
I would prefer a confidence interval for the difference between 2 ROC
areas to a P-value for the difference between 2 ROC areas. Such a
confidence interval can be computed using the -somersd- package
(downloadable from SSC) together with either the official Stata -lincom-
command or the SSC package -lincomest-. See Newson (2006), Newson (2003)
and Newson (2002) for details of how to do this.
However, having said this, it is important to stress that, if the
logistic models were fitted to the same set of data on which you are
estimating and testing the difference between the 2 ROC areas, then the
ROC areas for both logistic models will probably be over-optimistic, and
the ROC area for a logistic model with more parameters will probably be
more over-optimistic than the ROC area for a model with fewer
parameters. See Harrell et al. (1982) and Harrell et al. (1996) for some
methods for solving this problem, similar to bootstrapping and
cross-validation.
I hope this helps.
Roger
References
Harrell, F. E., R. M. Califf, D. B. Pryor, K. L. Lee, and R. A. Rosati.
Evaluating the yield of medical tests. Journal of the American Medical
Association 1982; 247(18): 2543-2546.
Harrell, F. E., K. L. Lee, and D. B. Mark. Multivariate prognostic
models: issues in developing models, evaluating assumptions and
adequacy, and measuring
and reducing errors. Statistics in Medicine 1996; 15: 361-387.
Newson R. Parameters behind "nonparametric" statistics: Kendall's tau,
Somers' D and median differences. The Stata Journal 2002; 2(1): 45-64.
Download pre-publication draft from
http://www.imperial.ac.uk/nhli/r.newson/
Newson R. Confidence intervals and p-values for delivery to the end
user. The Stata Journal 2003; 3(3): 245-269. Download pre-publication
draft from
http://www.imperial.ac.uk/nhli/r.newson/
Newson R. Confidence intervals for rank statistics: Somers' D and
extensions. The Stata Journal 2006; 6(3): 309-334. Download
pre-publication draft from
http://www.imperial.ac.uk/nhli/r.newson/
Roger Newson
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [email protected]
www.imperial.ac.uk/nhli/r.newson/
Opinions expressed are those of the author, not of the institution.
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of SR Millis
Sent: 05 June 2007 22:43
To: [email protected]
Subject: st: parametric v. nonparametric ROC curve analysis
I tested the difference in the area under ROC curves
for two different logistic models with the same sample
of patients. I did this using the logistic linear
predictors and the roccomp command, which uses the
algorithm suggested by DeLong et al. (1988) by
default.
A journal reviewer is questioning why I didn't use the
binormal method of Metz & Kronman (1980).
My reasoning is that I used a nonparametric ROC
analysis and didn't want to assume a bivariate normal
distribution. I suspect that both methods will yield
similar results but I would be interested in others'
viewpoints.
SR Millis
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
