RE: st: ANOVA Posthocs

["Airey, David C" <david.airey@vanderbilt.edu>]

.

I don't know of a Stata command that gives the exact p-values you want. The three UCLA commands tkcomp, fhcomp, and tukeyhsd report significance against a critical range statistic but not p-values <http://www.ats.ucla.edu/stat/stata/faq/pairwise.htm>.

net describe sg101, from(http://www.stata.com/stb/stb47) 
net describe tukeyhsd, from(http://www.ats.ucla.edu/stat/stata/ado/analysis) 
net describe tkcomp, from(http://www.ats.ucla.edu/stat/stata/ado/analysis) 
net describe fhcomp, from(http://www.ats.ucla.edu/stat/stata/ado/analysis) 
net describe dunnett, from(http://www.stata.com/users/tboswell/)

With smaller planned sets of tests, for FWER I like the simplicity of sequential bonferroni adjustments, like -test, mtest(holm)-. I guess when you get down to it, though, all these methods can be conservative, and can behave differently when there is unequal variance and unequal group size.

I also get the sense of an expectation with FDR methods, that you are using them to help guide additional data collection, rather than making hard claims without further evidence. My first reaction would be to submit uncorrected p-values to an FDR procedure, rather than try to correct them within each ANOVA before turning them into q-values. But that's a great question to bring up with some colleagues.

-Dave

> Yes David, we are going to run FDR through -multproc- on these t-tests as a second step. I should have mentioned this originally... 
> 
> Al hit on our larger philosophical question as well.  But either way I'd like to be able to capture p-values from post-hoc comparisons--whether Scheffe, Tukey HSD, Tukey-Kramer, or otherwise.
> 
> Rob
> 
> 

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/