Re: st: Testparm Command in Logistic Regression Analysis

[Alfonso Sanchez-Penalver <alfonso.statalist@gmail.com>]

The problem with having different number of observations is that in the one with more observations is that you are imposing also restrictions to have one model to fit the estimates to more observations, so when you need the likelihood from both specifications you don't know what the effect is oh the test statistic: the reduced coefficients or the additional observations. Also if the model with all the coefficients is the one with the more observations, then it is no longer clear which is the restricted and which is the unrestricted one.

So what to do? It is unclear which specification has the smaller number of observations, but -testparm- will test on the number of observations of the model with all the coefficients and -lrtest- with dropped observations on the smaller number of observations. If the difference in the number of observations is not large and you have a large sample, the two tests should be fine. In small samples it is not clear that testing in the reduced sample is a good idea, but you may not be left with a viable alternative.

I hope this helps,

Alfonso Sanchez-Penalver

> On Nov 27, 2013, at 8:45 AM, <isabel.lechner@vetsuisse.unibe.ch> wrote:
> 
> Thanks Marteen for your explanations, this helped me lot! 
> And yes, the different observations are due to missing values. Do I have to drop observations with missing values then or is there another way to account for them, so that I could perform the LR-Test with a max. number of observations?
> 
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Maarten Buis
> Sent: Mittwoch, 27. November 2013 14:16
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Testparm Command in Logistic Regression Analysis
> 
>> On Wed, Nov 27, 2013 at 1:54 PM,  <isabel.lechner@vetsuisse.unibe.ch> wrote:
>> Just in general, is it correct to use the test/testparm command in logistic regression or does it only apply for linear regression models?
> 
> -test- and -testparm- can be used after non-linear models, they will give you a Wald test.
> 
>> As I already stated I can't use the lrtest command to compare nested models because I have  different numbers of observations within the models.
> 
> The differing number of observation is probably due to missing values.
> If you take those into account you could use the likelihood ratio test.
> 
>> So what's the appropriate way to test the overall significance of a variable in a logistic regression model (and then to decide whether it stays in the model or not)?
> 
> Beware, in such a scenario you exclude variables because you cannot reject the hypothesis that the effect is 0. So, in essence you act as if a failure to reject the null hypothesis that the effect is 0 confirms that hypothesis. However, a failure to reject the that hypothesis is not the same as a confirmation of that hypothesis; an absence of evidence is not the same as evidence of absence. So, in a strict sense using any statistical test in this way is incorrect.
> 
> -- Maarten
> 
> ---------------------------------
> Maarten L. Buis
> WZB
> Reichpietschufer 50
> 10785 Berlin
> Germany
> 
> http://www.maartenbuis.nl
> ---------------------------------
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/