Hi Chris -
It doesn't appear that Stata will do post hoc tests on ranks for you automatically, but if you are willing to limit your post hoc tests to all pairwise comparisons, you can do each of them with Wilcoxon rank sum T-tests (equivalent to Mann-Whitney) tests, and use Bonferroni-corrected alphas for evaluating the results. In your case with 5 groups, you would to the 10 pairwise tests using "ranksum", and evaluate the resulting p-values at alpha' = .05/10 = .005. Any rank sum test with a p-value <= .005 would be judged significant. (See Marasuilo & McSweeney (1977), Nonparemetric and Distribution-Free Methods for the Social Sciences, p. 312, or their referenced article, Steel, R. G. D. (1960). A Rank Sum Test for Comparing All Pairs of Treatments. Technometrics, 2(2), 197-207.) If other post hoc comparisons are important for you (complex contrasts between, say, groups 1&2 versus 3), you are out of luck and will have to do hand calculations with the information provided by Stata. [
For several options, see Conover, W. J. (1999). Practical Nonparametric Statistics (3rd ed.). New York: Wiley, or Desu, M. M., & Raghavarao, D. (2004). Nonparametric Statistical Methods for Complete and Censored Data. Boca Raton, FL: Chapman & Hall/CRC.]
However, given that you have gone to a rank test only because you have a violation of the homogeneity of variance assumption, you might consider another approach. First, the use of Bartlett's test for homogeneity of variance is not the best approach, because it is known to be biased in the face of non-normality. I'm surprised that Stata still uses it, especially since they have three (!) better tests for equal variance with their "robvar" module, including Levene's test. So, it would be better first to check the homogeneity of variance assumption with "robvar" and if you reject the hypothesis of equal variances, then use an ANOVA procedure that does not assume equal variances.
Here again, however, Stata seems to be lacking. Although Stata provides a t-test for two independent means not assuming equal variances (actually, two versions: both the Satterthwaite's approximation and Welch's t-test), it does not provide an ANOVA F-test for the unequal variance situation. (At least, I have not been able to find this option.) This is very surprising, since it is a great program in so many other ways. Both SAS and SPSS (to name only two other widely used programs) provide tests when the homogeneity of variance assumption is not met. For example, SPSS provides both the Brown-Forsythe and the Welch F-tests in the ONEWAY program. Further, if the omnibus F-test is significant, then SPSS provides several post hoc multiple comparison procedures that do not assume equal variances, including the Dunnett's T3 procedure and Tamane's T2 procedure -- both of which are good options. So, if you have SPSS available, I'd recommend using it for this analysis. If your data is
not highly skewed (skewness < 2.0) and the groups are all greater than 10, then use the Welch F-test. If the groups are < 10, then use the Brown-Forsythe F-test. If the omnibus F-test is significant with (say) the Welch procedure, then you can do the post hoc comparisons with (say) Dunnett's T3. If you have SAS, you can obtain the omnibus Welch F-test, but I have been unable to find the corresponding post hoc tests that do not require equal variances. (For a good overview of these issues and tests, see Myers J. L., & Well, A. D. (2003). Research Design and Statistical Analysis (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.)
To summarize, if you are going to stay with Stata, then you can do all pairwise comparisons with ranksum and use Bonferroni-adjusted alphas. In this case, however, remember you cannot interpret your results on the original scale. If you want to stay with the original scale, then you'll have to use a program that will do the ANOVA not assuming equal variances. SPSS would be a good choice in this case, because it will do both the omnibus ANOVA F-test and the post hoc tests, not assuming equal variances.
Good luck!
bac
========================================
Bruce A. Cooper, PhD
Office of Research
UCSF School of Nursing
========================================
You said:
>Date: Tue, 3 Jan 2006 14:47:39 -0800 (PST)
>From: Chris Simpkins <[email protected]>
>Subject: st: Nonparametric Multiple Comparison Testing
>
>Is it possible to perform nonparametric multiple comparison testing in Stata? I am attempting to compare the distributions of a continuous variable (AGE) between five groups (group5) with unequal variances (based upon Bartlett's test using the oneway command) and I don't see any option for multiple comparison testing with the kwallis command. I searched through the Statalist archives and Stata's help but haven't been able to turn up any information on this.
>
>Here are the commands that I used:
>
>oneway AGE group5, bonferroni
>
>kwallis AGE, by(group5)
>
>Thanks for your help,
>
>Christopher Simpkins, M.D.
>Research Fellow
>Department of Surgery
>The Johns Hopkins University, School of Medicine
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
