RE: st: AW: What could be a potential difference between lroc after logistic and rocreg/roccurve

[Muhammad Riaz <riaz_stata@live.co.uk>]

Hi Martin,

Thank you, for reply.

Following to your advice I would like to go into more detailed querry, and produce the results on the basis of my data.

the command line and roctab result for AUC is given bellow

. roctab new_agg1yr HCRtot_a

ROC -Asymptotic Normal--
Obs Area Std. Err. [95% Conf. Interval]
--------------------------------------------------------
215 0.7084 0.0552 0.60017 0.81660

Now from bivariate logistic regression I get the

. logistic new_agg1yr HCRtot_a
Logistic regression Number of obs = 215
LR chi2(1) = 13.38
Prob> chi2 = 0.0003
Log likelihood = -72.598692 Pseudo R2 = 0.0843
------------------------------------------------------------------------------
new_agg1yr | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HCRtot_a | 1.128764 .0398917 3.43 0.001 1.053224 1.209722
------------------------------------------------------------------------------

the post estimation command give us

. lroc

Logistic model for new_agg1yr
number of observations = 215
area under ROC curve = 0.7084

We can see that the AUC is similar to the above results
------------------------------------------------------------------------------

But the rocreg give the following results


. rocreg new_agg1yr HCRtot_a
ROC regression for markers: Total HCR20 Time A
regression model covariates: none
percentile value calculation
method: empirical
tie correction: no
GLM fit of binormal curve
number of points: 10
on FPR interval: (0,1)
link function: probit
model coefficient bootstrap se's and CI's based on sampling
separately from cases and controls
number of bootstrap samples: 1000
******************************
model results for marker: Total HCR20 Time A
ROC-GLM model
Bootstrap results
Number of strata = 2 Number of obs = 221
Replications = 1000
------------------------------------------------------------------------------
| Observed Bootstrap
| Coef. Bias Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
alpha_0 | .72533184 -.0021167 .23319525 .2682776 1.182386 (N)
| .2810845 1.225096 (P)
| .303933 1.264812 (BC)
alpha_1 | .98818564 .0421458 .17240997 .6502683 1.326103 (N)
| .7120476 1.401654 (P)
| .6559572 1.309335 (BC)
------------------------------------------------------------------------------
(N) normal confidence interval
(P) percentile confidence interval
(BC) bias-corrected confidence interval


Which is over estimating AUC considering alpha_0= .72533184 is the value of AUC

------------------------------------------------------------------------------

Coparision of AUC's calculated from lroc and rocreg with covariate adjustment.


. logistic new_agg1yr HCRtot_a group site age readmissionyr1
Logistic regression Number of obs = 211
LR chi2(5) = 27.29
Prob> chi2 = 0.0001
Log likelihood = -65.121911 Pseudo R2 = 0.1732
------------------------------------------------------------------------------
new_agg1yr | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HCRtot_a | 1.123164 .0439389 2.97 0.003 1.040264 1.212671
group | 2.189092 1.02906 1.67 0.096 .8712182 5.500489
site | 1.205716 .207545 1.09 0.277 .8604467 1.68953
age | .9981258 .0243236 -0.08 0.939 .951573 1.046956
readmissio~1 | 3.998548 1.851404 2.99 0.003 1.61354 9.908887
------------------------------------------------------------------------------
. lroc
Logistic model for new_agg1yr
number of observations = 211
area under ROC curve = 0.7757
------------------------------------------------------------------------------


. rocreg new_agg1yr HCRtot_a, adjcov(group site age readmissionyr1) adjm(linear) cl( id)

ROC regression for markers: Total HCR20 Time A
regression model covariates: none
percentile value calculation
method: empirical
tie correction: no
Covariate adjustment for p.v. calculation:
method: linear model
covariates: psychiatric subgroups
site
age
whether or not readmitted in first year SM
GLM fit of binormal curve
number of points: 10
on FPR interval: (0,1)
link function: probit
model coefficient bootstrap se's and CI's based on sampling
separately from cases and controls
number of bootstrap samples: 1000
******************************
model results for marker: Total HCR20 Time A
covariate adjustment - linear model, controls only

Source | SS df MS Number of obs = 185
-------------+------------------------------ F( 4, 180) = 4.51
Model | 734.268759 4 183.56719 Prob> F = 0.0017
Residual | 7330.7907 180 40.726615 R-squared = 0.0910
-------------+------------------------------ Adj R-squared = 0.0708
Total | 8065.05946 184 43.8318449 Root MSE = 6.3817

------------------------------------------------------------------------------
HCRtot_a | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
group | -.5762791 1.012194 -0.57 0.570 -2.573572 1.421014
site | -1.014154 .3909454 -2.59 0.010 -1.785579 -.242728
age | -.0945426 .0454594 -2.08 0.039 -.1842445 -.0048407
readmissio~1 | 2.157065 1.122185 1.92 0.056 -.057264 4.371395
_cons | 24.40528 2.540936 9.60 0.000 19.39143 29.41914
------------------------------------------------------------------------------
************
ROC-GLM model
Bootstrap results
Number of strata = 2 Number of obs = 217
Number of clusters = 217
Replications = 1000
------------------------------------------------------------------------------
| Observed Bootstrap
| Coef. Bias Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
alpha_0 | .66002297 .0507902 .26150381 .1474849 1.172561 (N)
| .2443362 1.255396 (P)
| .1703629 1.189224 (BC)
alpha_1 | .92070132 .0846683 .19537493 .5377735 1.303629 (N)
| .6665187 1.448191 (P)
| .5577591 1.265389 (BC)
------------------------------------------------------------------------------
(N) normal confidence interval
(P) percentile confidence interval
(BC) bias-corrected confidence interval


comparing AUC (alpha_0 =.66002297) from rocreg to that computed from lroc AUC=0.7757 shows that rocreg under estimate with covariate adjustment even with option pvcm(normal) the result is alpha_0 =.70644182.

My question is can we use lroc for covariate adjustment in ROC analyses?
if we can not use this method then I need to use rocreg?
using rocreg, can I take the coeffiencent of alpha_0 as AUC?

If alpha_0 is AUC after the adjustment why it is different from the AUC produced by lroc for the same covariate adjustment (see analyses above) considering that we can use lroc for covariate adjstment in ROC analyses.

I will appreciate your help in this matter.

Best Regards

----------------------------------------
> From: martin.weiss1@gmx.de
> To: statalist@hsphsun2.harvard.edu
> Subject: st: AW: What could be a potential difference between lroc after logistic and rocreg/roccurve
> Date: Sun, 16 May 2010 16:29:08 +0200
>
>
> <>
>
> Make your post clearer by providing your code that leads to the
> discrepancies! I am not an expert in this area, but the fact that Pepe et
> al. went to great lengths to write their own commands and two articles in SJ
> 9(1) suggests to me that they address more involved questions than the ones
> that can be answered with -lroc- following a -logit-/-probit- model.
>
>
>
> HTH
> Martin
>
>
> -----Ursprüngliche Nachricht-----
> Von: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von Muhammad Riaz
> Gesendet: Sonntag, 16. Mai 2010 16:11
> An: statalist@hsphsun2.harvard.edu
> Betreff: st: What could be a potential difference between lroc after
> logistic and rocreg/roccurve
>
>
> Hi,
>
> I trying to perform adjusted ROC curve analyses.
> What could be a potential difference between (lroc) after logistic and
> (rocreg/roccurve)
> both commands produce different results for the same covariates.
> Regards,
>
> M
> _________________________________________________________________
> http://clk.atdmt.com/UKM/go/195013117/direct/01/
> We want to hear all your funny, exciting and crazy Hotmail stories. Tell us
> now
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/ 		 	   		  
_________________________________________________________________
http://clk.atdmt.com/UKM/go/197222280/direct/01/
We want to hear all your funny, exciting and crazy Hotmail stories. Tell us now
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/