1 item has been added to your cart.

How do I meta set my data when the effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs) with their confidence intervals?

The answer is similar, regardless of whether the effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs).

Let's assume that we want to perform meta-analysis using the following dataset, where the effect sizes are risk ratios (RRs). The confidence intervals for the risk ratios are also included in this dataset. Let’s have a look at the data.

. webuse bcgrratio2, clear
(BCG data with computed risk ratios)

. list rr rr_ll rr_ul


     
    rr   rr_ll   rr_ul 
  1. 
  .411    .134   1.257 
  2. 
  .205    .086    .486 
  3. 
   .26    .073    .919 
  4. 
  .237    .179    .312 
  5. 
  .804    .516   1.254 
  6. 
  .456    .387    .536 
  7. 
  .198    .078    .499 
  8. 
 1.012    .895   1.145 
  9. 
  .625    .393    .996 
 10. 
  .254    .149    .431 
 11. 
  .712    .573    .886 
 12. 
 1.562    .374   6.528 
 13. 
  .983    .582   1.659 

In order to perform meta-analysis, we need to use the meta set command to declare the meta-analysis data using the effect size variable and confidence interval variables.

Because meta set is designed to work with normally distributed effect sizes with symmetric confidence intervals, we cannot specify our rr, rr_ll, and rr_ul variables directly in meta set. As we can see here, the confidence intervals of RRs are not symmetric around RRs, because RRs are not normally distributed. The same happens with ORs, IRRs, and HRs. In order to get normalized effect sizes and symmetric confidence intervals, we need to apply the natural logarithm transformation to rr as well as the limits of the confidence intervals, rr_ll and rr_ul.

. generate double log_rr = log(rr)

. generate double log_rr_ll = log(rr_ll)

. generate double log_rr_ul = log(rr_ul)

. list log_rr log_rr_ll log_rr_ul


     
     log_rr    log_rr_ll    log_rr_ul 
  1. 
 -.88916206   -2.0099155    .22872793 
  2. 
 -1.5847453    -2.453408   -.72154666 
  3. 
 -1.3470736   -2.6172958   -.08446916 
  4. 
 -1.4396951   -1.7203695   -1.1647521 
  5. 
 -.21815601   -.66164851    .22633844 
  6. 
 -.78526247   -.94933059   -.62362112 
  7. 
 -1.6194882   -2.5510465   -.69514918 
  8. 
  .01192857   -.11093156    .13540464 
  9. 
 -.47000363   -.93394567   -.00400802 
 10. 
  -1.370421    -1.903809   -.84164719 
 11. 
 -.33967737   -.55686956   -.12103833 
 12. 
  .44596705   -.98349948    1.8761006 
 13. 
 -.01714616   -.54128483    .50621501 

In the above code, we specify the storage type double when we generate the transformed variables because we want them to have a high precision. double has a higher precision than the default float type (please see [D] Data types for more detailed information). Next we specify meta set with the transformed data. However, we get the following error message:

. meta set log_rr log_rr_ll log_rr_ul
confidence intervals not symmetric
  CIs defined by variables log_rr_ll and log_rr_ul must be
  symmetric and based on a normal distribution. If you are working with
  effect sizes in the original metric, such as odds ratios or hazard
  ratios, with meta set, you should specify the effect sizes and
  CIs in a normalizing metric, such as the log metric.

  The default tolerance to determine the CI asymmetry is 1e-6. Effect
  sizes and their CIs are often reported with limited precision that,
  after the normalizing transformation, may lead to asymmetric CIs. In
  that case, the default of 1e-6 may be too stringent. You may loosen
  the tolerance by specifying option civartolerance().
r(459);

This error message indicates that meta set determined that the confidence intervals are not symmetric. Why are the transformed confidence intervals still not symmetric? The original variables rr, rr_ll, and rr_ul are usually stored with only a few decimal digits (three digits in this case), and therefore the transformed confidence intervals are slightly asymmetric around transformed effect sizes. You should not see this error message if your data have adequate precision.

By default, the symmetry check is performed by meta set with a tolerance level of 1e-6, which means the largest relative difference between log_rr_ul-log_rr and log_rr-log_rr_ll cannot be greater than 1e-6. (The relative difference between \(x\) and \(y\) is defined as |\(x\)–\(y\)|/(|\(y\)|+1).) Let’s compute the relative difference for our transformed dataset.

. generate double rel_diff = reldif((log_rr_ul-log_rr), (log_rr-log_rr_ll))

. summarize rel_diff


    Variable 
 
        Obs        Mean    Std. dev.       Min        Max
 
 
 
    rel_diff 
 
         13    .0019658    .0013932   .0002746   .0044752

We can see the maximum relative difference is 0.0044752, which is greater than 1e-6. That is why the symmetry check failed. We also notice the values of relative difference are all reasonably small, which confirms that the natural logarithm transformation is appropriate here (to make RRs normal). If the relative difference has any large values, that means the transformed confidence intervals are not close to symmetric. In this case, you need to check how the effect sizes and their confidence intervals are originally computed and decide if a different type of transformation is needed.

In the current situation, where the transformed confidence intervals are roughly symmetric, we can relax the criterion for the symmetry check by specifying the option civartolerance() with meta set. The default tolerance level for the symmetry check is 1e-6. Now we can change it to 1e-2 to avoid this error. The bigger the tolerance level we specify, the less strict the performed check will be. If you are interested, please see Why am I getting the error message “confidence intervals not symmetric” after I run meta set? for more detailed information about this error message and the option civartolerance().

. meta set log_rr log_rr_ll log_rr_ul, civartolerance(1e-2)

This time, the symmetry check is passed and we have declared the meta-analysis data using the transformed variables. We are ready for some analysis.

If we run meta summarize or meta forestplot, we will see that, by default, log RRs are displayed as the effect size instead of RRs because this is the data metric specified with meta set.

. meta summarize

  Effect-size label: Effect size
        Effect size: log_rr
          Std. err.: _meta_se

Meta-analysis summary                     Number of studies =     13
Random-effects model                      Heterogeneity:
Method: REML                                          tau2 =  0.3121
                                                    I2 (%) =   92.20
                                                        H2 =   12.82



            Study 
 
    Effect size    [95% conf. interval]  % weight

 
 
 
 

         Study  1 
 
         -0.889      -2.008       0.230      5.05
 

         Study  2 
 
         -1.585      -2.451      -0.719      6.36
 

         Study  3 
 
         -1.347      -2.613      -0.081      4.42
 

         Study  4 
 
         -1.440      -1.718      -1.162      9.71
 

         Study  5 
 
         -0.218      -0.662       0.226      8.87
 

         Study  6 
 
         -0.785      -0.948      -0.622     10.11
 

         Study  7 
 
         -0.785      -0.948      -0.622     10.11
 

         Study  8 
 
          0.012      -0.111       0.135     10.20
 

         Study  9 
 
         -0.470      -0.935      -0.005      8.75
 

         Study 10 
 
         -1.370      -1.902      -0.839      8.36
 

         Study 11 
 
         -0.340      -0.558      -0.122      9.94
 

         Study 12 
 
          0.446      -0.984       1.876      3.82
 

         Study 13 
 
         -0.017      -0.541       0.507      8.41
 

 
 
 
 

            theta 
 
         -0.714      -1.066      -0.362          
 


Test of theta = 0: z = -3.97                     Prob > |z| = 0.0001
Test of homogeneity: Q = chi2(12) = 151.72         Prob > Q = 0.0000

However, you might prefer to display the effect sizes in the original metric (RRs). In that case, you can specify the option eform(Risk ratios) with meta summarize or meta forestplot.

. meta summarize, eform(Risk ratios)

  Effect-size label: Effect size
        Effect size: log_rr
          Std. err.: _meta_se

Meta-analysis summary                     Number of studies =     13
Random-effects model                      Heterogeneity:
Method: REML                                          tau2 =  0.3121
                                                    I2 (%) =   92.20
                                                        H2 =   12.82



            Study 
 
    Risk ratios    [95% conf. interval]  % weight

 
 
 
 

         Study  1 
 
          0.411       0.134       1.259      5.05
 

         Study  2 
 
          0.205       0.086       0.487      6.36
 

         Study  3 
 
          0.260       0.073       0.923      4.42
 

         Study  4 
 
          0.237       0.180       0.313      9.71
 

         Study  5 
 
          0.804       0.516       1.253      8.87
 

         Study  6 
 
          0.456       0.387       0.537     10.11
 

         Study  7 
 
          0.198       0.078       0.501      6.01
 

         Study  8 
 
          1.012       0.895       1.145     10.20
 

         Study  9 
 
          0.625       0.393       0.995      8.75
 

         Study 10 
 
          0.254       0.149       0.432      8.36
 

         Study 11 
 
          0.712       0.573       0.885      9.94
 

         Study 12 
 
          1.562       0.374       6.526      3.82
 

         Study 13 
 
          0.983       0.582       1.660      8.41
 

 
 
 
 

       exp(theta) 
 
          0.490       0.345       0.696          
 


Test of theta = 0: z = -3.97                     Prob > |z| = 0.0001
Test of homogeneity: Q = chi2(12) = 151.72         Prob > Q = 0.0000

. meta forestplot, eform(Risk ratios)

The option eform() displays the exponentiated effect sizes and the exponentiated confidence intervals using the label we specify, therefore showing our data in the original (RRs) metric. Please note that when eform() is specified, the x axis is automatically shown on an exponentiated scale that makes confidence intervals look symmetric, which is the standard in the literature.