Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: How to compare RMSE of two models
From
"JVerkuilen (Gmail)" <[email protected]>
To
[email protected]
Subject
Re: st: How to compare RMSE of two models
Date
Tue, 27 Nov 2012 09:14:13 -0500
On Tue, Nov 27, 2012 at 7:49 AM, rasool.bux <[email protected]> wrote:
> Dear Maarten,
>
> Thanks for reply. May be I am not conveying my question clearly. I would like to compare the two models against the same yvariable, which model is performing better and how we could say that model 1 is better than model 2 i.e. in terms of p-values of RMSE.
>
I assume you don't have nested models, because otherwise you would
simply use -testparm- to consider whether the additional coefficients
are jointly 0. Non-nested model tests are tricky. The Vuong test is
one example. http://en.wikipedia.org/wiki/Vuong's_closeness_test
However, if your y variables are the same (i.e., no missing data) you
can probably get most of what you want using AIC or by comparing
adjusted R-square changes between models. If you must have a p-value I
suspect that the only reasonable way is to use bootstrapping:
http://stata.com/statalist/archive/2010-03/msg01845.html
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/