Thanks Ken,
Getting that issued straightened out let me complete a program
to "line" multiple comparison results in the classical manner.
The sample below shows the output for the example data shown in
my previous message. Here, the lines under "Groups" indicate
groups (ordered by mean value) of treatments that do not differ
significantly from each other. Thus there are two groups, AE-AD
and AD-AC, wherein the treatments do not differ significantly
from each other based on the Sidak-adjusted p. By definition,
treatment AF differs from all other treatments, as it is not
included in a group.
.. mc_lines vol, by(id) sidak sd
Multiple comparison t-tests at p = .05, adjustment method: Sidak
vol: Volume
----------+----------+-----+-----------
Mean : SD : ID : Groups
----------+----------+-----+-----------
856.4 : 183.51 : AF :
931.3 : 193.51 : AE : |
934.4 : 207.72 : AA : |
953.3 : 201.67 : AB : |
1003.5 : 212.35 : AD : | |
1049.2 : 232.39 : AC : |
----------+----------+-----+-----------
As others have before me, I believe this is a much more useful
and interpretable presentation format for such data.
An initial version is available to any Statalisters who would be
willing to test it and provide feedback; just drop me a note. It will
do lining for Sidak-, Bonferroni- and Scheffe-adjusted, as well as
unadjusted multiple t-tests from a oneway design.
Tom Steichen
Ken Higbee <[email protected]> replied in part:
> > I have noted on a number of occasions that the Bonferroni and Sidak
> > adjusted p-values from multiple comparisons by the -oneway- command
> > differ slightly (but more than simple rounding error) from what my
> > hand calculations suggest.
> >
> > Comparison of Volume by ID
> > (Sidak)
> > Row Mean-|
> > Col Mean | AA AB AC AD AE
> > ---------+-------------------------------------------------------
> > AB | 18.9015
> > | 1.000
> > |
> > AC | 114.777 95.8757
> > | 0.000 0.002
> > |
> > AD | 69.0773 50.1758 -45.7
> > | 0.092 0.494 0.655
> > |
> > AE | -3.07991 -21.9814 -117.857 -72.1572
> > | 1.000 0.999 0.000 0.057
> > |
> > AF | -77.9796 -96.8811 -192.757 -147.057 -74.8997
> > | 0.023 0.001 0.000 0.000 0.030
> >
> > If someone can confirm this problem -- or point out the
> > error in my calculations -- I would appreciate it.
>
> You are using 132+132-2 as the degrees of freedom. However,
> s=sqrt(within MS) is computed from the pooled groups (all 6
> groups in your case) and the degrees of freedom for that pooled s
> is 823. So for your example, to get the Sidak adjusted p-value,
> you would
>
> . di min(1, 1-(1-ttail(823,
> 69.0773/(sqrt(42178.2188)*sqrt(1/132+1/132)))*2)^15)
> .09208965
>
> which agrees with what -oneway- produced for the Sidak adjusted
> p-value for the comparison of AA and AD. Likewise for the
> Bonferroni adjusted p-value, you need to use 823 instead of
> 132+132-2 for the degrees of freedom in your call to -ttail()-.
> For Scheffe you were already using the 823 degrees of freedom in
> the call to -Ftail()-. That is why it was in agreement with
> -oneway-s output.
-----------------------------------------
CONFIDENTIALITY NOTE: This e-mail message, including any
attachment(s), contains information that may be confidential,
protected by the attorney-client or other legal privileges, and/or
proprietary non-public information. If you are not an intended
recipient of this message or an authorized assistant to an intended
recipient, please notify the sender by replying to this message and
then delete it from your system. Use, dissemination, distribution,
or reproduction of this message and/or any of its attachments (if
any) by unintended recipients is not authorized and may be
unlawful.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
