--- On Mon, 3/8/09, Jason Ferris wrote:
> Stata 10 released Fisher's exact methods for small sample
> studies. These methods are for non-linear (logistic,
> poisson) distributions and apply to RxC based categorical
> models. I would like to know if there exists a more
> robust way of analysing small sample pilot studies (i.e.,
> 10 people in each treatment group: treatment and placebo)
> where the outcome is continuous? That is, something
> similar to Stata's exact statistics.
Those exact methods exist because everything gets harder
when your model is non-linear in the parameters. On the
positive side: everything gets easier when things are
linear in the parameters, for instance when you do a
simple t-test. Alternatively, if you are worried about the
normality assumption, you could look at the whole range of
statistics based on rank statistics, for instance Roger
Newson's somersd package (Newson 2002, 2006a 2006b). You
can get the somersd by typing in Stata:
-ssc install somersd-. If you have such a small sample size
then your first port of call should probably be to plot the
individual observations. I like Nick Cox's -stripplot- for
that, to get it you type in Stata -ssc install stripplot-.
The helpfile contains lots of useful examples you can run
using the auto dataset (the dataset you get when you type
in Stata -sysuse auto-. I would not go for the so called
robust standard errors you can get in many regression type
of analyses, as their logic is based on large samples.
As an aside I am not so sure that exact methods are so
suitable for small samples, as they aren't exact at all but
conservative. Remember that if you have choosen a
significance level of 5% than that means that you want to
reject a true null hypothesis in 5% of the samples you draw.
With exact methods you will do so less often. That sounds
good, but a) it means that exact methods aren't exact at all,
and b) the price you pay is less power for your statistical
test (less likely to find a significant difference when you
should) If you have very small datasets then you need to be
as efficient as possible with the little information you do
have, and not throw some power away because the method has a
nice catchy name like exact methods.
Hope this helps,
Maarten
Roger Newson, 2006a, Confidence intervals for rank
statistics: Somers' D and extensions. The Stata Journal,
6(3): 309-334.
http://www.stata-journal.com/article.html?article=snp15_6
Roger Newson, 2006b, Confidence intervals for rank
statistics: Percentile slopes, differences, and ratios.
The Stata Journal, 6(4):497-520.
http://www.stata-journal.com/article.html?article=snp15_7
Roger Newson, 2002, Parameters behind "nonparametric"
statistics: Kendall's tau, Somers' D and median
differences. The Stata Journal, 2(1): 45-64.
http://www.stata-journal.com/article.html?article=st0007
-----------------------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany
http://home.fsw.vu.nl/m.buis/
-----------------------------------------
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
