Non-rejection definition:
A 95% confidence interval, (L, U), consists of all values of theta that
can not be rejected at the 5% significance level given the data.
An exact confidence region defined in that way will not always be an
interval if the test statistic is based on a discrete random variable, eg
in the case of Fisher's exact test, because there may be "holes" in the
non-rejection region, caused by the fact that the P-value can only take
finitely many values (or maybe countably infinitely many values as in the
Poisson case). The conservative confidence intervals defined by
Clopper-Pearson, Mehta-Patel-Gray etc. include the holes, and are not exact
either in the coverage sense or in the non-rejection sense, although they
are conservative in the coverage sense. However, they are exact in that
they use the exact discrete distribution of the test statistic, instead of
a continuous approximation.
