Re: st: stcox with shared frailty: curse of dimensionality?

[rgutierrez@stata.com (Roberto G. Gutierrez, StataCorp)]

Priscila Ferreira <prisferr@gmail.com> asks:

> I tried to estimate a cox model with shared frailty. The number of groups is
> greater than 100.000. Such specification does not run (matsize too small is
> the error message). However, everything goes alright if I use streg with
> shared frailty.

> From the manual I couldn't go very far on this issue - I suppose that if we
> can retrieve estimates of the log frailties, they are being stored in
> memory. Consequently, the matsize problem appears. If this is the correct
> argument, I don't understand why such estimates are being stored in stcox
> and not in streg (because in both of them a parametric distribution is
> assumed for the frailties).

Priscilla's assertion that it is a curse of dimensionality is correct.  Cox
models with shared frailties are fitted using penalized likelihood, where the
log-frailities are estimated in conjunction with the regression parameters.
Parametric models, on the other hand, have the benefit that the distribution
of the frailities can combine with that for the response and thus the
frailties are not directly estimated, but instead integrated out of the
likelihood.

Priscilla is correct that in both models a distribution is assumed for the
frailities.  It's just that with Cox regression you are not dealing with the
distribution of time-to-failure, but instead with a marginal distribution of
failure ranks, for which a direct extension to include frailties (whatever
their distribution) that may be integrated out does not exist.

--Bobby
rgutierrez@stata.com
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/