Re: st: RE: Goodness of fit

[Mikkel Brabrand <mikkel@brabrand.net>]

Thanks,

The thing is that HL is a postestimation command and, when validating, I will not run a new regression... This is why I believe that I need to calculate it by hand.

Mikkel

Den 03/05/2011 kl. 15.20 skrev Visintainer, Paul:

> There is a free-standing version of the Hosmer-Lemeshow test that can be found by typing:
> 
> .net from http://www.sealedenvelope.com/
> 
> From within Stata.
> 
> I think what you want to do is to take the model (your scoring system) developed from your development dataset, and apply it to the relevant variables in the validation dataset.  This will generate the expected mortality in the validation set given your scoring system.  You can use -roctab- to see how well your scoring system discriminates the cases in the validation set.  The H-L test will evaluate model fit.  SMRs can be used, but all from the validation set (the "observed" is your outcomes from the validation set and your "expected" is the risk score from the validation set).
> 
> The H-L test is a chi-square with 2 degrees of freedom and usually based on decile cutpoints.
> 
> Paul 
> 
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of mikkelbrabrand
> Sent: Tuesday, May 03, 2011 8:03 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: st: Goodness of fit
> 
> I have developed a risk scoring system that I'm trying to validate. It was
> originally developed in one cohort, and I am now validating it in a new,
> independent, cohort. However, I am unsure how to test for goodness-of-fit
> using Hosmer-Lemeshow's test.
> 
> First question: When validating my score, is it not correct that I should
> use
> the observed mortality (my endpoint) from the development cohort to estimate
> the expected mortality in my validation cohort?
> 
> Second question: When calculating the Chi-square between the observed and
> expected mortality, should my formula not be: (observed-expected)^2/
> (expected*1-expected/N (in this stratum))?
> 
> Third question: How many degrees of freedome should I use? When I read the
> book by Hosmer and Lemeshow (Applied logistic regression), in the section on
> validation in an external cohort, I understand that the number of groups
> equal the degrees of freedom, which in different from the ordinary method in
> which the degrees of freedom is groups-2. Is this correct?
> 
> Thanks!
> Mikkel 
> 
> --
> View this message in context: http://statalist.1588530.n2.nabble.com/Goodness-of-fit-tp6327041p6327041.html
> Sent from the Statalist mailing list archive at Nabble.com.
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 
> ----------------------------------------------------------------------
> Please view our annual report at http://baystatehealth.org/annualreport
> 
> 
> CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please reply to the sender immediately or by telephone at 413-794-0000 and destroy all copies of this communication and any attachments. For further information regarding Baystate Health's privacy policy, please visit our Internet site at http://baystatehealth.org.
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/