Pairwise comparisons

Stata has two commands for performing all pairwise comparisons of means and other margins across the levels of categorical variables. The pwmean command provides a simple syntax for computing all pairwise comparisons of means. After fitting a model with almost any estimation command, the pwcompare command can perform pairwise comparisons of estimated marginal means and other types of marginal linear predictions. In addition, the margins command allows for performing all pairwise comparisons of linear and nonlinear predictions, such as marginal probabilities. With each of these commands, p-values and confidence intervals can be adjusted for multiple comparisons.

To obtain all pairwise differences of the mean of y across the levels of a treatment and adjust the p-values and confidence intervals for multiple comparisons using Tukey’s HSD, we can type

. pwmean y, over(treatment) mcompare(tukey) effects

Pairwise comparisons of means with equal variances

Over: treatment





             
 
    Number of

             
 
  comparisons

 
 
 

   treatment 
 
           10






             
 
                              Tukey                Tukey

           y 
 
   Contrast   Std. err.      t    P>|t|     [95% conf. interval]

 
 
 

   treatment 
 
 

     2 vs 1  
 
    3.62272   1.589997     2.28   0.156    -.7552913    8.000731

     3 vs 1  
 
   .4906299   1.589997     0.31   0.998    -3.887381    4.868641

     4 vs 1  
 
   4.922803   1.589997     3.10   0.019     .5447922    9.300815

     5 vs 1  
 
  -1.238328   1.589997    -0.78   0.936    -5.616339    3.139683

     3 vs 2  
 
   -3.13209   1.589997    -1.97   0.285    -7.510101    1.245921

     4 vs 2  
 
   1.300083   1.589997     0.82   0.925    -3.077928    5.678095

     5 vs 2  
 
  -4.861048   1.589997    -3.06   0.021    -9.239059   -.4830368

     4 vs 3  
 
   4.432173   1.589997     2.79   0.046     .0541623    8.810185

     5 vs 3  
 
  -1.728958   1.589997    -1.09   0.813    -6.106969    2.649053

     5 vs 4  
 
  -6.161132   1.589997    -3.87   0.001    -10.53914    -1.78312

If treatment=1 is a control and the other levels represent treatments, we may want to use Dunnett’s method for making comparisons.

. pwmean y, over(treatment) mcompare(dunnett) effects

Pairwise comparisons of means with equal variances

over         : treatment





             
 
    Number of

             
 
  comparisons

 
 
 

   treatment 
 
            4






             
 
                             Dunnett              Dunnett

           y 
 
   Contrast   Std. err.      t    P>|t|     [95% conf. interval]

 
 
 

   treatment 
 
 

     2 vs 1  
 
    3.62272   1.589997     2.28   0.079    -.2918331    7.537273

     3 vs 1  
 
   .4906299   1.589997     0.31   0.994    -3.423923    4.405183

     4 vs 1  
 
   4.922803   1.589997     3.10   0.008      1.00825    8.837356

     5 vs 1  
 
  -1.238328   1.589997    -0.78   0.852    -5.152881    2.676225

After fitting a model, we can use pwcompare to make pairwise comparisons of the margins. We could fit the fully interacted model

. regress y treatment##grp

and obtain pairwise comparisons of all the cell means for the interaction.

. pwcompare treatment#grp, group

Pairwise comparisons of marginal linear predictions

Margins: asbalanced




              
 
                         Unadjusted

              
 
     Margin   Std. err.      groups

 
 
 

treatment#grp 
 
 

         1 0  
 
   36.91257   1.116571        A    

         1 1  
 
   45.81229   1.116571         B   

         2 0  
 
   38.79482   1.116571        A C  

         2 1  
 
   51.17547   1.116571            E

         3 0  
 
   36.34383   1.116571        A    

         3 1  
 
   47.36229   1.116571         B   

         4 0  
 
   41.81757   1.116571          CD 

         4 1  
 
    50.7529   1.116571            E

         5 0  
 
   35.69507   1.116571        A    

         5 1  
 
   44.55313   1.116571         B D 


Note: Margins sharing a letter in the group label
      are not significantly different at the 5%
      level.

Because there are many pairwise comparisons, we obtain the results of the tests symbolically. Two means that have the same letter are not significantly different from each other at a 5% significance level.