Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: choosing between two non nested models
From
"Craig, Benjamin M." <[email protected]>
To
<[email protected]>
Subject
RE: st: choosing between two non nested models
Date
Wed, 1 Sep 2010 08:44:15 -0400
This is an impossible question, because of its dependence on your
personal perspective.
If you were a Bayesian, Bayes factors should decide model selection;
however, BIC is proportional to the factors in large samples, and is
much easier to estimate.
If you are frequentist, it is messier. AIC is entirely arbitrary in its
trade-off between bias and variance, but it is justifiable if you agree
with its normative assumptions. Alternatively, if you just care about
prediction, you could apply out-of-sample predictions techniques (see
Copas test).
As a last resort, frequentists may try re-sampling techniques, where
errors from one model are incorporated with predictions from another to
construct psuedo data to see if the model results are numerically
nested, even if their formulae are not (e.g., logit vs. probit). These
more complicated techniques require greater sophistication for the
analyst and audience than may be required.
As an econometrician, I recommend that you state your model first, then
estimate. Any other model is an exploratory analysis, and rejecting a
model using data may take a sample size beyond that available. Theory
should drive your model selection, not data. This is no right or wrong
answer, just evidence.
Cheers,
Ben
Benjamin M. Craig, PhD
Assistant Member, Health Outcomes & Behavior, Moffitt Cancer Center
Associate Professor, Economics, University of South Florida
12902 Magnolia Dr, MRC-CANCONT, Tampa, FL 33612-9416
Phone (813) 745-6710; Fax (813) 745-6525
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Richard
Williams
Sent: Wednesday, September 01, 2010 8:55 AM
To: [email protected]; [email protected]
Subject: Re: st: choosing between two non nested models
At 05:32 AM 9/1/2010, Michael N. Mitchell wrote:
>I wonder if AIC and/or BIC may be of help? You can see
>
>. help estat ic
>
>for more information. I think this is a longshot, but hope it may pay
off.
I don't think those stats are legitimate if the observations are
different. But I wonder why they are different in the first place? I
think you would usually want to be analyzing the same cases if you want
to compare two models. It would be easy enough to do a listwise
deletion of all cases that have missing data in the two models.
-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
OFFICE: (574)631-6668, (574)631-6463
HOME: (574)289-5227
EMAIL: [email protected]
WWW: http://www.nd.edu/~rwilliam
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
This transmission may be confidential or protected from disclosure and is only for review and use by the intended recipient. Access by anyone else is unauthorized. Any unauthorized reader is hereby notified that any review, use, dissemination, disclosure or copying of this information, or any act or omission taken in reliance on it, is prohibited and may be unlawful. If you received this transmission in error, please notify the sender immediately. Thank you.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/