Re: st: multproc

["Roger B. Newson" <r.newson@imperial.ac.uk>]

No, you are not doing anything incorrectly. However, as stated in Newson 
et al. (2003), the two Benjamini-Liu methods sometimes have the annoying 
feature that the corrected P-value is lower than the uncorrected P-value.

I personally usually use the Simes-Benjamini-Hochberg procedure 
(-method(simes)-), rather than the Liu procedures. And, nowadays, I 
usually use -qqvalue- instead of -multproc-, because I prefer q-values 
to discovery sets. The use of q-values is probably an advance on 
discovery sets, because, given the q-values, every individual in the 
audience can input his/her own FDR and define his/her own discovery set. 
For more about -qqvalue- and frequentist q-values, see Newson (2010).

I hope this helps.

Best wishes

Roger

References

Newson R and the ALSPAC Study Team. Multiple-test procedures and smile 
plots. The Stata Journal 2003; 3(2): 109-132. Download from
http://www.stata-journal.com/article.html?article=st0035

Newson RB. Frequentist q-values for multiple-test procedures. The Stata 
Journal 2010; 10(4): 568-584. Download from
http://www.stata-journal.com/article.html?article=st0209

Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology, Occupational Medicine
and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7594 7931
Email: r.newson@imperial.ac.uk
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page: http://www.imperial.ac.uk/nhli/reomph/

Opinions expressed are those of the author, not of the institution.

On 19/03/2014 23:25, Char Adams wrote:
> Hello,
>
>
> Does anyone have experience with false discovery methods in Stata?  I asked StataHelp and was told there wasn't an official command.
>
> I downloaded smileplot, but I don't know if I'm using multproc correctly.  I've varied the choice of method within the syntax I'm trying and I don't see much of difference between methods (bonferroni verus simes etc....)
>
> I've got a dataset which provides the pvalues in a column with my tests of interest, but I was expecting to see a corrected pvalue that would be less stringent than what I get with bonferroni.  Instead, it is about the same, depending on the method.
>
> I may not be using the code correctly.
>
>
> Here's the code I've tried:
> multproc ,method(liu1) puncor(.05)
>
> What I get:
> Method: liu1
> Uncorrected overall critical P-value: .05
> Number of P-values: 391
> Corrected overall critical P-value: .00013118
> Number of rejected P-values: 0
>
> Am I doing something incorrectly?  Is there any easier way?
>
> Thank you for your time,
> Charleen
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/