RE: st: one way anova or kruskal wallis if sample size is less than 3 for each group

[Nick Cox <n.j.cox@durham.ac.uk>]

Although it may be obvious, I'd add the comment that a graph is likely to show essentially all the structure in this data set. 

If you work through what Mann-Whitney U-tests actually mean when n = 2 or 3, your results are less surprising. 

Quantifying the underlying probabilities through use of Roger Newson's -somersd- may also be revealing. 

Bonferroni, (W.H.) Kruskal, Wallis, Mann and Whitney are all represented by vignettes in the Stata on-line documentation. 

Nick 
n.j.cox@durham.ac.uk 

Morten Hesse

In principle, a problem for ANOVA is when variance and mean are  
correlated (i.e., typically that at higher mean scores, variance is  
also higher). The Bartlett test shows a trend towards different  
variances, which could indicate this.

This is an instance where you may justify using Kruskal-Wallis.
But what I would like to ask, is why you would even bother to run  
statistical tests on something that is so obviously different. If you  
can show a difference with 2-3 cases per cell, the difference is  
likely to be so obvious that nobody would bother to question it. You  
would not run statistical analyses to test whether 18-year old humans  
are taller than 3-year old children, simply because it is obvious that  
there will be a difference.

However, if you still feel that you need to justify your restults  
through statistical analyses, I would recommend reporting the ANOVA,  
the Bonferroni post-hoc, and mention that you have tried the KW, and  
it gave significant results.

<redacted>

> I have a data with 5 groups (group) and a quantitative dependent  
> variable (score). Two groups have sample size of two and 4 groups  
> have sample size of 3.
>
> I run Kruskal-Wallis test and because the P-value is less than 0.05,  
> I do Mann-Whitney test. I am baffled that none of the p value from  
> Mann-Whitney test is less than 0.05. I also run one-way anova with  
> Bonferroni correction, and the results show p values of less than  
> 0.05.
>
> Can someone explain this and which test should I choose?
>
> Thank you very much
>
>
> KRUSKAL WALLIS
>
> . kwallis score, by(group)
>
> Kruskal-Wallis equality-of-populations rank test
>
> +------------------------+
> | group | Obs | Rank Sum |
> |-------+-----+----------|
> | 1 | 2 | 29.00 |
> | 2 | 2 | 3.00 |
> | 3 | 3 | 12.00 |
> | 4 | 3 | 21.00 |
> | 5 | 3 | 41.00 |
> |-------+-----+----------|
> | 6 | 3 | 30.00 |
> +------------------------+
>
> chi-squared = 14.309 with 5 d.f.
> probability = 0.0138
>
> chi-squared with ties = 14.330 with 5 d.f.
> probability = 0.0136
>
> . ranksum score if group == 1 | group ==2, by(group)
>
> Two-sample Wilcoxon rank-sum (Mann-Whitney) test
>
> group | obs rank sum expected
> -------------+---------------------------------
> 1 | 2 7 5
> 2 | 2 3 5
> -------------+---------------------------------
> combined | 4 10 10
>
> unadjusted variance 1.67
> adjustment for ties 0.00
> ----------
> adjusted variance 1.67
>
> Ho: score(group==1) = score(group==2)
> z = 1.549
> Prob > |z| = 0.1213
>
> *** other Mann-Whitney results not shown
>
>
> ONE WAY ANOVA
>
> . oneway score group, bonferroni
>
> Analysis of Variance
> Source SS df MS F Prob > F
> ------------------------------------------------------------------------
> Between groups 43047.0833 5 8609.41667 106.20 0.0000
> Within groups 810.666667 10 81.0666667
> ------------------------------------------------------------------------
> Total 43857.75 15 2923.85
>
> Bartlett's test for equal variances: chi2(5) = 9.2622 Prob>chi2 = 0.099
>
> Comparison of score by group
> (Bonferroni)
> Row Mean-|
> Col Mean | 1 2 3 4 5
> ---------+-------------------------------------------------------
> 2 | -133
> | 0.000
> |
> 3 | -125 8
> | 0.000 1.000
> |
> 4 | -111.667 21.3333 13.3333
> | 0.000 0.400 1.000
> |
> 5 | -8 125 117 103.667
> | 1.000 0.000 0.000 0.000
> |
> 6 | -70 63 55 41.6667 -62
> | 0.000 0.000 0.000 0.003 0.000
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/