 |
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
RE: st: gologit2
> > I <[email protected]> wrote:
>> I would suggest instead approaching this specification problem by
>> looking at the relative increase in the pseudo-R^2 value associated
>> with moving to a non-proportional odds model...
.....snip, snip
and
Maarten buis <[email protected]> thoughtfully responded
>
>If you want to go along this route, I would personally be more
>confortable with comparing BICs rather than pseudo R^2s, as BICs where
>actually designed for comparing across models and there is an
>approximate justification for what kind of difference is large. For
>those who care: they can be turned into an approxemation of the
>posterior odds ratio of one model versus the other given uniform
>priors, see (Raftery 1995)
>
>- -- Maarten
>
>Raftery, Adrian E. (1995). Bayesian model selection in social research
>(with Discussion). Sociological Methodology, 25, 111-196.
I haven't tried the BICs in this context, but when I used them for
model selection problems with a large sample and a binary response
model, I found the BIC to be quite sensitive to trivial differences.
The specific situation that comes to mind was a comparison of models with
and without several interaction terms, and I found the BIC indicating the
superiority of the more complete model even though the differences in modeled
slopes and predicted probabilities were quite small. Also, I was stumped
by the standard advice to use some fixed difference in BIC value
as an indicator of a better model even though, other things equal, this
difference depends on sample size. So, whatever might be said about the BIC
in terms of a better theoretical foundation, it did not seem to offer an
alternative to the mixing of size of effect and size of the sample that
characterizes conventional hypothesis testing, and so did not help much
with the compromise between parsimony and exactness.
I'd be interested to see how others (i.e., people more sophisticated
about the BIC :-}) would see these kinds of issues.
Regards,
=-=-=-=-=-=-=-=-=-=-=-=-=
Mike Lacy, Assoc. Prof.
Soc. Dept., Colo. State. Univ.
Fort Collins CO 80523 USA
(970)-491-6721
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/