Thanks to Bobby for letting us know the current position re vector
degrees of freedom in Stata.
Re Al's query, I don't know of any results about degrees of freedom
formulas in -xtmixed-. However, in the case of the Satterthwaite degrees
of freedom with the 2-sample t-test, the Satterthwaite degrees of
freedom formula seems to work like a dream. The definitive study is that
of Moser et al. (1989) and Moser and Stevens (1992), which was a
numerical integration study (rather than a simulation study), and which
tested the Satterthwait unequal-variance and equal-variance t-test
confidence intervals under between-subpopulation variance ratios from 1
by 1 to 10 and pairs of sample sizes from thye set {6,11,51}. Under all
conditions tested, the Satterthwaite formula gave coverage percentages
within 1 percentage point of the advertized value of 95%. The
equal-variance t-test gave confidence intervals that were too small if
the smaller sample was from the more variable subpopulation, too large
if the larger sample was from the more variable subpopulation, and with
essentially the correct coverage probability if either the subpopulation
variances or the sample numbers were equal. The advantage of the
equal-variance formula is that, if the population variances are the same
and the sample sizes different, then we can estimate the population
variability of the smaller sample using the sample variability of the
larger sample, and thereby gain a bit of power by reducing the
confidence interval width. The authors therefore seem to view the
Satterthwaite formula as the recommended default formula, and to view
the equal-variance formula as a special formula for the special
occasions when we think that we really can estimate the population
variability of the smaller sample using the sample variability of the
larger sample. They advise against the common practice of using the
equal-variance formula unless a heteroskedasticity test indicates
otherwise, essentially because heteroskedasticity may affect the
validity of confidence intervals without being detectable by a
heteroskedasticity test.
I have done a simulation study on the model of the numerical integration
study of Moser et al. with a view to testing to destruction my own
-cendif- command, which is part of the -somersd- package and estimates
robust confidence intervals for Hodges-Lehmann median differences. I
have found the Satterthwaite formula to work as advertized, at least if
the data are sampled from Normal distributions. Unsurprisingly, it is
possible to break the Satterthwaite formula by feeding it with data
sampled from Cauchy distributions, which are very prone to extreme
values, which expand the standard error more than they throw the sample
mean difference and therefore cause major loss of power. I am currently
writing my study up for publication.
I hope this helps.
Best wishes
Roger
References
Moser BK, Stevens GR, Watts CL. The two-sample t-test versus
Satterthwaite's approximate F-test. Communications in Statistics -
Theory and Methods 1989; 18(11): 3963-3975.
Moser BK, Stevens GR. Homogeneity of variance in the two-sample means
test. The American Statistician 1992; 46(1): 19-21.
Roger Newson
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [email protected]
www.imperial.ac.uk/nhli/r.newson/
Opinions expressed are those of the author, not of the institution.
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Feiveson,
Alan H. (JSC-SK311)
Sent: 11 April 2007 21:21
To: [email protected]
Subject: RE: st: Vector degrees of freedom in Stata
Roger, Stas, Bobby (or anyone else)
Are there any results (published or unofficial) that suggest that
computing P-values using a t-distribution with pseudo degrees of freedom
consistently gives more correct inference with -xtmixed- than simple
Wald tests? Or is this not what you guys are talking about?
Al Feiveson
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Roberto G.
Gutierrez, StataCorp
Sent: Wednesday, April 11, 2007 3:09 PM
To: [email protected]
Subject: Re: st: Vector degrees of freedom in Stata
Roger Newson <[email protected]> inquires about vectorizing
degrees of freedom calculations in -xtmixed-, and other Stata estimation
commands for that matter:
[...]
> My query is as follows. Does StataCorp have plans, especially plans
> that can be revealed, for implementing vector degrees of freedom in
> estimation commands? And, if so, will there be a convention for
> storing the vector degrees of freedom as an estimation result? (For
> instance, the vector degrees of freedom might be stored in a row
> vector with length colsof(e(b)), named e(df_vec).) I ask because I am
> considering the possibility of introducing vector degrees of freedom
> for some of my commands, and would like my programs to use similar
terminology to StataCorp's if possible.
While we have thought of doing this with -xtmixed-, as of this time such
consideration would be specific to only that command. As such, you are
unlikely to stomp on any Stata general convention for how these things
are handled. Who knows, we may even decide to go with your way.
--Bobby
[email protected]
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
