Re: st: Area under the ROC curve less than 0.5?

[Roger Newson <r.newson@imperial.ac.uk>]

This is a common problem. The solution is to exit Stata after installing 
-somersd- and to launch Stata again. The -somersd- package should then work.

The reason for this problem is that -somersd- uses a Mata library, 
containing a suite of Mata programs. When Stata is launched, it creates 
a list of Mata libraries in which to look for Mata programs. If the user 
then installs a package with a Mata library (such as -somersd-), then 
some versions of Stata will not update this list by adding the new Mata 
library to the list. However, if the user exits Stata and then launches 
Stata again, then the list of Mata libraries will be created again, and 
should include the Mata library in the -somersd- package.

I hope this helps. Let me know if it doesn't.

Best wishes

Roger


Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: r.newson@imperial.ac.uk
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/

Opinions expressed are those of the author, not of the institution.

On 24/01/2011 04:33, reggae.benigno@metrobank.com.ph wrote:
> Thank you very much Professor for taking the time to answer my queries. It really helped me comprehend better how AROC works. I also did as you suggested, and was able to install the -somersd- package by typing in "ssc install somersd". However, when I tried using the program, stata returned an error message:
>
> tidotforsomersd():  3499  panelsetup() not found
> 	<istmt>:     -  function returned error
>
> Can you also help me figure out how to fix this?
>
>
> Very much obliged,
>
>
> Reggae
>
>
>
>
>  From :   Roger Newson<r.newson@imperial.ac.uk>
> To : "statalist@hsphsun2.harvard.edu"<statalist@hsphsun2.harvard.edu>
> Subject : Re: st: Area under the ROC curve less than 0.5?
> Date :  Fri, 21 Jan 2011 13:54:56 +0000
>
> In reply to your first query, the area under the ROC can be anything from 0 to 1. Predictors with a ROC area less than 0.5 are negative prwedictors, and predictors with a ROC area between 0.5 and 1 are positive predictors.
>
> In reply to your second query, it is possible for the difference between 2 ROC areas to be statistically non-significant in a small sample, even if one ROC area is less than half the other. This is because the confidence interval for the difference between 2 ROC areas may be wide, and include a zero difference and a range of positive and negative differences. To get a confidence interval for the difference between 2 ROC areas (also known as Harrell's c statistics), use the -somersd- package, which you can download from SSC and which has 3 .pdf manuals, distributed with the package as ancillary files. The on-line help contains hyper-references to the Stata Journal articles explaining the -somersd- package, which you can use with -lincom- to get confidence intervals for the difference between 2 Harrell's c statistics.
>
> I hope this helps.
>
> Best wishes
>
> Roger
>
>
> Roger B Newson BSc MSc DPhil
> Lecturer in Medical Statistics
> Respiratory Epidemiology and Public Health Group
> National Heart and Lung Institute
> Imperial College London
> Royal Brompton Campus
> Room 33, Emmanuel Kaye Building
> 1B Manresa Road
> London SW3 6LR
> UNITED KINGDOM
> Tel: +44 (0)20 7352 8121 ext 3381
> Fax: +44 (0)20 7351 8322
> Email: r.newson@imperial.ac.uk
> Web page: http://www.imperial.ac.uk/nhli/r.newson/
> Departmental Web page:
> http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/
>
>
> Opinions expressed are those of the author, not of the institution.
>
> 	On 21/01/2011 09:56, reggae.benigno wrote:
>
> 	Hi!
>
> 	I just want to ask, after running a logistic regression on a 	development sample, I ran the lroc command. The corresponding area 	under  the ROC curve is .9226. When I wished to check the robustness 	of the model on a holdout sample, I keyed in "lroc if sample==2". The 	software was able to run it on this separate sample, however, the 	resulting AROC is now .4060. I thought AROC values are only from 0.5 	to 1.0, or was I mistaken?
>
> 	Also, when I tried running roccomp on the two samples, Stata was again
> 	able to churn out the same ROCs (.9226 and .4060), but the chi-square
> 	p-value is .0663, indicating that at alpha=.05, the null hypothesis of
> 	equal areas under the curves cannot be rejected.  Does it make sense
> 	that even though the AROC of the second sample is less than half of 	the first sample, the test still says that the 2 ROC curves have equal
> 	areas? Is this test perhaps affected by the number of observations?
> 	(sample 1 : 26 positive, 174 negative, while sample 2 contains 5(+) 	and 102(-)).
>
> 	Would really appreciate anyone's help. Thanks in advance!
>
>
> 	- Reggae
>
>
>
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/