Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | Muhammad Riaz <riaz_stata@live.co.uk> |
| To | <statalist@hsphsun2.harvard.edu> |
| Subject | RE: st: AW: What could be a potential difference between lroc after logistic and rocreg/roccurve |
| Date | Tue, 18 May 2010 06:59:58 +0100 |
Hi Martin, Thank you, for reply. Following to your advice I would like to go into more detailed querry, and produce the results on the basis of my data. the command line and roctab result for AUC is given bellow . roctab new_agg1yr HCRtot_a ROC -Asymptotic Normal-- Obs Area Std. Err. [95% Conf. Interval] -------------------------------------------------------- 215 0.7084 0.0552 0.60017 0.81660 Now from bivariate logistic regression I get the . logistic new_agg1yr HCRtot_a Logistic regression Number of obs = 215 LR chi2(1) = 13.38 Prob> chi2 = 0.0003 Log likelihood = -72.598692 Pseudo R2 = 0.0843 ------------------------------------------------------------------------------ new_agg1yr | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- HCRtot_a | 1.128764 .0398917 3.43 0.001 1.053224 1.209722 ------------------------------------------------------------------------------ the post estimation command give us . lroc Logistic model for new_agg1yr number of observations = 215 area under ROC curve = 0.7084 We can see that the AUC is similar to the above results ------------------------------------------------------------------------------ But the rocreg give the following results . rocreg new_agg1yr HCRtot_a ROC regression for markers: Total HCR20 Time A regression model covariates: none percentile value calculation method: empirical tie correction: no GLM fit of binormal curve number of points: 10 on FPR interval: (0,1) link function: probit model coefficient bootstrap se's and CI's based on sampling separately from cases and controls number of bootstrap samples: 1000 ****************************** model results for marker: Total HCR20 Time A ROC-GLM model Bootstrap results Number of strata = 2 Number of obs = 221 Replications = 1000 ------------------------------------------------------------------------------ | Observed Bootstrap | Coef. Bias Std. Err. [95% Conf. Interval] -------------+---------------------------------------------------------------- alpha_0 | .72533184 -.0021167 .23319525 .2682776 1.182386 (N) | .2810845 1.225096 (P) | .303933 1.264812 (BC) alpha_1 | .98818564 .0421458 .17240997 .6502683 1.326103 (N) | .7120476 1.401654 (P) | .6559572 1.309335 (BC) ------------------------------------------------------------------------------ (N) normal confidence interval (P) percentile confidence interval (BC) bias-corrected confidence interval Which is over estimating AUC considering alpha_0= .72533184 is the value of AUC ------------------------------------------------------------------------------ Coparision of AUC's calculated from lroc and rocreg with covariate adjustment. . logistic new_agg1yr HCRtot_a group site age readmissionyr1 Logistic regression Number of obs = 211 LR chi2(5) = 27.29 Prob> chi2 = 0.0001 Log likelihood = -65.121911 Pseudo R2 = 0.1732 ------------------------------------------------------------------------------ new_agg1yr | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- HCRtot_a | 1.123164 .0439389 2.97 0.003 1.040264 1.212671 group | 2.189092 1.02906 1.67 0.096 .8712182 5.500489 site | 1.205716 .207545 1.09 0.277 .8604467 1.68953 age | .9981258 .0243236 -0.08 0.939 .951573 1.046956 readmissio~1 | 3.998548 1.851404 2.99 0.003 1.61354 9.908887 ------------------------------------------------------------------------------ . lroc Logistic model for new_agg1yr number of observations = 211 area under ROC curve = 0.7757 ------------------------------------------------------------------------------ . rocreg new_agg1yr HCRtot_a, adjcov(group site age readmissionyr1) adjm(linear) cl( id) ROC regression for markers: Total HCR20 Time A regression model covariates: none percentile value calculation method: empirical tie correction: no Covariate adjustment for p.v. calculation: method: linear model covariates: psychiatric subgroups site age whether or not readmitted in first year SM GLM fit of binormal curve number of points: 10 on FPR interval: (0,1) link function: probit model coefficient bootstrap se's and CI's based on sampling separately from cases and controls number of bootstrap samples: 1000 ****************************** model results for marker: Total HCR20 Time A covariate adjustment - linear model, controls only Source | SS df MS Number of obs = 185 -------------+------------------------------ F( 4, 180) = 4.51 Model | 734.268759 4 183.56719 Prob> F = 0.0017 Residual | 7330.7907 180 40.726615 R-squared = 0.0910 -------------+------------------------------ Adj R-squared = 0.0708 Total | 8065.05946 184 43.8318449 Root MSE = 6.3817 ------------------------------------------------------------------------------ HCRtot_a | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- group | -.5762791 1.012194 -0.57 0.570 -2.573572 1.421014 site | -1.014154 .3909454 -2.59 0.010 -1.785579 -.242728 age | -.0945426 .0454594 -2.08 0.039 -.1842445 -.0048407 readmissio~1 | 2.157065 1.122185 1.92 0.056 -.057264 4.371395 _cons | 24.40528 2.540936 9.60 0.000 19.39143 29.41914 ------------------------------------------------------------------------------ ************ ROC-GLM model Bootstrap results Number of strata = 2 Number of obs = 217 Number of clusters = 217 Replications = 1000 ------------------------------------------------------------------------------ | Observed Bootstrap | Coef. Bias Std. Err. [95% Conf. Interval] -------------+---------------------------------------------------------------- alpha_0 | .66002297 .0507902 .26150381 .1474849 1.172561 (N) | .2443362 1.255396 (P) | .1703629 1.189224 (BC) alpha_1 | .92070132 .0846683 .19537493 .5377735 1.303629 (N) | .6665187 1.448191 (P) | .5577591 1.265389 (BC) ------------------------------------------------------------------------------ (N) normal confidence interval (P) percentile confidence interval (BC) bias-corrected confidence interval comparing AUC (alpha_0 =.66002297) from rocreg to that computed from lroc AUC=0.7757 shows that rocreg under estimate with covariate adjustment even with option pvcm(normal) the result is alpha_0 =.70644182. My question is can we use lroc for covariate adjustment in ROC analyses? if we can not use this method then I need to use rocreg? using rocreg, can I take the coeffiencent of alpha_0 as AUC? If alpha_0 is AUC after the adjustment why it is different from the AUC produced by lroc for the same covariate adjustment (see analyses above) considering that we can use lroc for covariate adjstment in ROC analyses. I will appreciate your help in this matter. Best Regards ---------------------------------------- > From: martin.weiss1@gmx.de > To: statalist@hsphsun2.harvard.edu > Subject: st: AW: What could be a potential difference between lroc after logistic and rocreg/roccurve > Date: Sun, 16 May 2010 16:29:08 +0200 > > > <> > > Make your post clearer by providing your code that leads to the > discrepancies! I am not an expert in this area, but the fact that Pepe et > al. went to great lengths to write their own commands and two articles in SJ > 9(1) suggests to me that they address more involved questions than the ones > that can be answered with -lroc- following a -logit-/-probit- model. > > > > HTH > Martin > > > -----Ursprüngliche Nachricht----- > Von: owner-statalist@hsphsun2.harvard.edu > [mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von Muhammad Riaz > Gesendet: Sonntag, 16. Mai 2010 16:11 > An: statalist@hsphsun2.harvard.edu > Betreff: st: What could be a potential difference between lroc after > logistic and rocreg/roccurve > > > Hi, > > I trying to perform adjusted ROC curve analyses. > What could be a potential difference between (lroc) after logistic and > (rocreg/roccurve) > both commands produce different results for the same covariates. > Regards, > > M > _________________________________________________________________ > http://clk.atdmt.com/UKM/go/195013117/direct/01/ > We want to hear all your funny, exciting and crazy Hotmail stories. Tell us > now > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ _________________________________________________________________ http://clk.atdmt.com/UKM/go/197222280/direct/01/ We want to hear all your funny, exciting and crazy Hotmail stories. Tell us now * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/