1 item has been added to your cart.

Why am I getting the error message “confidence intervals not symmetric” after I run meta set? How can I avoid this error?

meta set has two main syntax specifications. One is for specifying generic effect sizes with their standard errors, and the other one is for specifying generic effect sizes with their confidence intervals. When the second syntax is used, you may encounter the error of “confidence intervals not symmetric”.

Consider the following fictional meta-analysis dataset, where es is the effect size and cil and ciu are the confidence interval lower point and upper point, respectively.

clear
input es    cil    ciu
1.48  -.35  3.31
1     -.93  2.93
1.27  .43   2.12
1     .75   1.25
1.18  -.53  2.88
1.94  .43   3.45
2.38  1.01  3.75
.69   -.57  1.96
1.1   -.15  2.34
1.81  -.15  3.76
end

Running meta set gives you the following error:

. meta set es cil ciu
confidence intervals not symmetric
    CIs defined by variables cil and ciu must be symmetric and based on a normal
    distribution. If you are working with effect sizes in the original metric, such as
    odds ratios or hazard ratios, with meta set, you should specify the effect sizes and
    CIs in a normalizing metric, such as the log metric.

    The default tolerance to determine the CI asymmetry is 1e-6. Effect sizes and their
    CIs are often reported with limited precision that, after the normalizing
    transformation, may lead to asymmetric CIs. In that case, the default of 1e-6 may be
    too stringent. You may loosen the tolerance by specifying option civartolerance().
r(459);

This error indicates that the confidence interval we specified is not symmetric around the effect size for at least one study in the dataset. As explained in the error message, meta set assumes that confidence intervals for effect sizes are based on the normal distribution, and therefore they should be symmetric. Mathematically, the effect sizes and symmetric confidence intervals should satisfy

CI upper limit – effect size = effect size – CI lower limit

There are two situations that may produce asymmetric CIs, leading meta set to issue an error.

First situation

In the first situation, confidence intervals for the effect sizes are indeed based on the normal distribution, but they are recorded with less precision than meta set expects. In this situation, confidence intervals will look roughly symmetric around the effect sizes. For example, you might have collected data from a publication that only gives you two decimal digits, and therefore they might not pass the command's symmetry check. By default, the symmetry check is performed with a tolerance level of 1e-6. That means the largest relative difference between CI upper limit – effect size and effect size – CI lower limit cannot be greater than 1e-6. (The relative difference between \(x\) and \(y\) is defined as |\(x\)–\(y\)|/(|\(y\)|+1).)

Let’s examine our data for this purpose.

. generate double relative_diff = reldif(es-cil, ciu-es)

. summarize relative_diff


    Variable 
 
        Obs        Mean    Std. dev.       Min        Max
 
 
 
relative_d~f 
 
         10    .0021369    .0023118          0   .0054054

We can see the largest relative difference is about .0054, which exceeds 1e-6; that is why the symmetry check failed. Because the confidence intervals in this data are roughly symmetric around effect sizes, we know this is the first situation. In this case, you could consider using the option civartolerance() with meta set to relax the symmetry check for confidence intervals. We already mentioned that the default setting for this check is civartolerance(1e-6). We could change that number to be slightly bigger than the largest relative difference we obtain above. For example, we can use

. meta set es cil ciu,civartolerance(0.0055)

Here we specify 0.0055 in civartolerance(), and this time the symmetry check is passed.

When you specify civartolerance() with meta set, Stata will obtain a symmetric confidence interval based on the old confidence interval width for each study in your original data. The new confidence intervals are saved in the variables _meta_cil and _meta_ciu, and they will be used throughout the meta-analysis (which explains why you may get a forest plot with confidence intervals that slightly differ from the original ones in your dataset).

The new confidence intervals satisfy

_meta_ciu – _meta_cil = ciu – cil    (The CI width remains the same)

_meta_ciu – es = es – _meta_cil       (symmetric CI)

Second situation

In the second situation, the effect sizes are in a metric for which normality is not expected, and the confidence intervals are not close to symmetric. You may find the dataset requires specifying a relatively big number in civartolerance() in order to run meta set without error. This is usually indicative of potential problems beyond the number of significant digits reported. Perhaps typos were introduced when collecting the data, or perhaps the data are not reported in a normalizing metric and you may need to apply a normalizing transformation to obtain symmetric intervals. For example, if your effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs), you should apply a logarithmic transformation to normalize your data.

For more details on this process, please see How do I meta set my data when the effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs) with their confidence intervals?.