RE: st: ANOVA Posthocs

["Ploutz-Snyder, Robert (JSC-SK)[USRA]" <robert.ploutz-snyder-1@nasa.gov>]

Thank you Dave.  I think I'm at the same point as you on this one... submitting uncorrected p-values to -multproc- for FDR, as your assumption was correct--this is a first-step towards identifying "promising" genes for further inquiry.  

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Airey, David C
Sent: Sunday, October 24, 2010 11:45 PM
To: statalist@hsphsun2.harvard.edu
Subject: RE: st: ANOVA Posthocs

.

I don't know of a Stata command that gives the exact p-values you want. The three UCLA commands tkcomp, fhcomp, and tukeyhsd report significance against a critical range statistic but not p-values <http://www.ats.ucla.edu/stat/stata/faq/pairwise.htm>.

net describe sg101, from(http://www.stata.com/stb/stb47) 
net describe tukeyhsd, from(http://www.ats.ucla.edu/stat/stata/ado/analysis) 
net describe tkcomp, from(http://www.ats.ucla.edu/stat/stata/ado/analysis) 
net describe fhcomp, from(http://www.ats.ucla.edu/stat/stata/ado/analysis) 
net describe dunnett, from(http://www.stata.com/users/tboswell/)

With smaller planned sets of tests, for FWER I like the simplicity of sequential bonferroni adjustments, like -test, mtest(holm)-. I guess when you get down to it, though, all these methods can be conservative, and can behave differently when there is unequal variance and unequal group size.

I also get the sense of an expectation with FDR methods, that you are using them to help guide additional data collection, rather than making hard claims without further evidence. My first reaction would be to submit uncorrected p-values to an FDR procedure, rather than try to correct them within each ANOVA before turning them into q-values. But that's a great question to bring up with some colleagues.

-Dave

> Yes David, we are going to run FDR through -multproc- on these t-tests as a second step. I should have mentioned this originally... 
> 
> Al hit on our larger philosophical question as well.  But either way I'd like to be able to capture p-values from post-hoc comparisons--whether Scheffe, Tukey HSD, Tukey-Kramer, or otherwise.
> 
> Rob
> 
> 

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/