Re: st: Goodness of Fit Measure for Generalized Linear Models with Adjustment for the Number of Parameters

[Nick Cox <njcoxstata@gmail.com>]

The impulse here is a little puzzling to me. Others here will have a
deeper mathematical statistics grasp of this than I do, but as I think
no one has commented I will jump in.

The model you're fitting is estimated using a pseudo- or quasi-maximum
likelihood procedure. That doesn't rule out calculating an R-squared
measure as a descriptive or heuristic indicator of goodness of fit,
which I've been positive about elsewhere e.g.
http://www.stata.com/support/faq/statistics/r-squared/index.html That
is a stretch insofar as your fitting is strictly not equivalent to
maximising R-squared, which is one view of regression. But as long as
you use words like "heuristic" people may not be too harsh about that.

However, if you now consider the general idea that you should consider
penalising yourself for using several predictors, the impulse to
adjust R-squared seems even more of a stretch. If there is a need to
think about the trade-off between simplicity and fit it is perhaps
better done using AIC or BIC.

Note that everything is at least a little controversial in this
territory: most people are moderately fond of some information
criterion, but there is essentially no agreement that one is best.
Nick
njcoxstata@gmail.com


On 4 March 2014 17:04, Roberto Liebscher <roberto.liebscher@ku.de> wrote:
> Hello Statalisters,
>
> I model a fractional response variable with a GLM similar to Papke, L.E.,
> Wooldridge, J.M., 1996. Econometric Methods for Fractional Response
> Variables with an Application to 401(K) Plan Participation Rates. Journal of
> Applied Econometrics 11 (1). 619-632.
>
> I would like to obtain a goodness-of-fit measure that incorporates the
> number of parameters in a fashion similar to the adjusted R-squared. It is
> tempting to compute the correlation between the predicted and the observed
> values (like in Christopher F Baum's example here:
> http://fmwww.bc.edu/EC-C/S2013/823/EC823.S2013.nn06.slides.pdf ) and compute
> the adjusted R-squared according to the formula
> $R^2-(1-R^2)\frac{p}{n-p-1}$. Since I have never seen something similar in
> papers so far my question is if there is something wrong about it?
>
> Moreover, from a computational point of view one could also estimate the
> quasi log likelihood function of the unrestricted and the restricted model
> and follow McFadden's procedure (McFadden's adjusted R^2:
> http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm ). If
> the only goal is to compare non-nested models is there any reason not to use
> such a measure?
>
> Any help is highly appreciated.
>
> Roberto
>
> --
> Roberto Liebscher
> Catholic University of Eichstaett-Ingolstadt
> Department of Business Administration
> Chair of Banking and Finance
> Auf der Schanz 49
> D-85049 Ingolstadt
> Germany
> Phone:  (+49)-841-937-1929
> FAX:    (+49)-841-937-2883
> E-mail:   roberto.liebscher@ku-eichstaett.de
> Internet: http://www.ku.de/wwf/lfb/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/