Re: st: AIC for comparing probit and logit

[Maarten Buis <maartenlbuis@gmail.com>]

On Fri, Jul 6, 2012 at 4:58 PM, Laurie Molina wrote:
> Do you know if it is valid to compare between and probit and logit
> model, by comparing the corresponding AICs?
> Is AIC valid to compare models with different distributions? or does
> is it only valid to compare models with the same distribution, for
> example two logit models with different explanatory variables?

Could be possible, but there are many things that could go wrong. In
particular, many likelihood functions typical contain a part that does
not depend on the parameters, and programmers can choose to keep those
in or leave those out without changing the parameter estimates.
However, such choices do influence whether or not the AICs can be
compared.

The real difficulty is that I would only believe the result if the
conclusion is that -probit- and -logit- are to all intends and
purposes identical. If you look at the curves you'll see that they are
virtually identical (logit having a slightly thicker tail). You'd need
a huge dataset to detect such differences, and when you do, these
differences will have no practical implication whatsoever. If all that
is true, than there is just no need to test it... You can choose
whichever model is prefered by your (sub-(sub-))discipline.

Hope this helps,
Maarten

--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany


http://www.maartenbuis.nl
--------------------------
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/