Re: st: Collect and combine CIs of ROC curves from bootstrapped datasets

[Steve Samuels <sjsamuels@gmail.com>]

Haluk

With multiple bootstrap samples, you combine data sets, not
confidence intervals.  If you have already created the
 data sets,  -append- into one larger one and then run -bstat-. 
Some of the -bstat- results are available only in matrices, e.g.

. matrix list e(ci_bc)  // for bias-corrected CIs

This advice was available to you in the Manual.

If you are starting from scratch, then instead of doing, say, 10
bootstrap runs of B = 100 replicates each; do one run of B = 1,000
replicates instead. Follow -bootstrap- by -estat bootstrap, all-.

I strongly recommend that you read the Altman reference below.
There is Stata code for cross-validation in
http://www.epibiostat.ucsf.edu/courses/roadmapk12/sigs/prediction.pdf.

Reference: G Altman, Vergouwe Yvonne, Royston Patrick, and G M Moons
Karel. 2009. Prognosis and prognostic research: validating a prognostic
model. BMJ 338, available http://www.bmj.com/content/338/bmj.b605.

Steve

Steven J. Samuels
18 Cantine's Island
Saugerties NY USA


On Nov 28, 2013, at 9:53 AM, Haluk Vahaboglu wrote:

Dear Stata listers,
I am using Stata 12.1 for Linux 64.
I am trying to generate a predictive scale for a binary outcome (coded as 1/0). My dataset includes 507 patients with 169 events. Eventually, I selected independent variables from a logistic model and developed a scale by giving ranks to eight variables.
To validate internally the performance of this scale, I generate ROC curve and compare AUC to the gold standard.
Also I want to estimate limits of CIs by collecting and combining CIs from for example 10 rounds of bootstrapped datasets.
For this purpose, I use -rocreg- command of Stata or a user written ado file -bsrintercept-  (by H. Glick, http://www.uphs.upenn.edu/dgimhsr/stat-roc.htm).
My question is, can I generate these bootstrapped datasets iteratively and collect AUCs and CIs by means of a loop (or whatever) and combine these.
I am unfamiliar with the jargon used among statisticians and additionally not native. Probably it would be very difficult to understand what I am trying to say. However, I am in desperate need for a bit of help and would be grateful.

Haluk Vahaboğlu
Inf Dis & Clin Micro


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/