Re: st: Generating logistic regression models for forecasting

[Richard Goldstein <richgold@ix.netcom.com>]

I have no idea what crossval.ado is or where it can be found

however, in my opinion, unless N>20,000, you are better off using
-bootstrap-

Rich

On 1/8/14, 5:16 PM, Brent McSharry (ADHB) wrote:
> Dear statalisters
> 
> The question is more of a statistics than a Stata question, so apologies if this is the wrong forum (please feel free to redirect me to a more appropriate forum).
> 
> Years ago, when developing logistic regression models for forecasting, data would be randomly split into development and testing data sets (using something like .gen develop = runiform() < 1/3). This is much less popular now, and using Rick Thompson's crossval.ado I have a very useful tool for testing fit and discrimination of variables included in the final model.
> 
> However, in the stages of model development, It would be logical that I should no longer limit included data to a randomly defined development data set (eg logit x y z if develop). The whole point of the old technique (to my understanding) was to avoid overfitting when choosing which covariates to include. Is it reasonable to develop the model on the whole data set, and then test the final model using crossval, or should I be limiting the data set used for model development some how (or indeed use some tool for multiple resampling to obtain model coefficients and estimates, while in the model development phase)?
> 
> Thank you very much.
> 
> Brent McSharry MBBS BSc(med) FCICM(paed)
> Paediatric Intensivist
> Starship Children's Hospital
> Private Bag 92024
> Auckland 1142
> New Zealand 
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/