Re: st: positive interaction - negative covariance

[David Hoaglin <dchoaglin@gmail.com>]

The problems that arise from trying to compare confidence intervals
are more general.  They arise in situations where the estimates are
independent.  Thus, the covariance in the sampling distribution of b1
and b3 is not the real issue.

To assess the difference between two estimates, it is usually a
mistake to compare their confidence intervals.  The correct approach
is to form the appropriate confidence interval for the difference and
ask whether that confidence interval includes zero.  I often encounter
people who think that they can determine whether two estimates (e.g.,
the means of two independent samples) are different by checking
whether the two confidence intervals overlap.  They are simply wrong.
The article by Schenker and Gentleman (2001) explains.  (I said
"usually" above to exclude intervals that are constructed specifically
for use in assessing the significance of pairwise comparisons.)

David Hoaglin

Nathaniel Schenker and Jane F. Gentleman, On judging the significance
of differences by examining the overlap between confidence intervals.
The American Statistician 2001; 55(3):182-186.

On Sat, Feb 23, 2013 at 10:50 AM, Maarten Buis <maartenlbuis@gmail.com> wrote:
>>>>>> Thank you for clarifying the point. Indeed, if I compare the
>>>>>> confidence interval of b1 to the confidence interval of (b1+b3), they
>>>>>> are not statistically different. Is it the same? In a book I read, the
>>>>>> authors make this comparison.
>
> That is a common mistake. If you want to know whether the two curves
> are equal you must look at the interaction term (b3), not whether the
> confidence intervals of both curves (b1 and b1 + b3) overlap. The
> latter ignores the covariance in the sampling distribution of both
> parameters, and is thus wrong. See for example:
>
> Andrew Gelman and Hal Stern (2006) "The Difference Between
> `Significant' and `Not Significant' is not Itself Statistically
> Significant" The American Statistician, 60(4):328--331.
>
> -- Maarten

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/