Re: st: Statistical significance vs the 95% confidence intervals-- how should i interpret these

[Richard Goldstein <richgold@ix.netcom.com>]

that CIs do not overlap does not mean that the estimates are "not 
statistically significantly different".  they can overlap and still be
statistically significantly different -- the test uses the SE of the
difference which is different from the SEs used for the CIs

This is discussed in many places in the stat literature; a simple
discussion can be found in Wolfe, R and Hanley, J (1/8/2002), "If we're so
different, why do we keep overlapping? When 1 plus 1 doesn't make 2",
_Canadian Medical Association Journal_, 166: 65-66.

Rich Goldstein

Columbia & Belmont Apt wrote:

> Dear Statalist Colleagues,
> This might be a simple question but I cannot reconcile the following results.
>
> The question is "Are the estimates for year 1 and 3 statistically different?
> year    estimate    st.error        95% conf interval 
> year1    .484721     .015637     .454068    .5153741   
> year2   .4128893    .0145616    .3843443    .4414343    
> year3   .4556235    .0138994    .4283765    .4828704    
> 1. Using the 95% confidence interval that I obtained:
>
> Year 1 and year 3 confidence intervals overlap thus year 1&3 estimates are not statistically different at the 95% level.
>
> 2. Using ttesti they are statistically different with 99% confidence: 
> . ttesti 2398 .484721  .015637 2399 .4556235 .0138994
> Two-sample t test with equal variances
> ------------------------------------------------------------------------------
>          |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
> ---------+--------------------------------------------------------------------
>        x |    2398     .484721    .0003193     .015637    .4840948    .4853472
>        y |    2399    .4556235    .0002838    .0138994     .455067      .45618
> ---------+--------------------------------------------------------------------
> combined |    4797    .4701692    .0002996    .0207488    .4695819    .4707565
> ---------+--------------------------------------------------------------------
>     diff |            .0290975    .0004272                  .02826     .029935
> ------------------------------------------------------------------------------
> Degrees of freedom: 4795
>                       Ho: mean(x) - mean(y) = diff = 0
>      Ha: diff < 0               Ha: diff != 0              Ha: diff > 0
>        t =  68.1143                t =  68.1143              t =  68.1143
>    P < t =   1.0000          P > |t| =   0.0000          P > t =   0.0000
>
>
> Which one is the correct? What am I not interpreting right?
>
> Thank you in advance.
> Sincerely,
> Alla
>
>
>  
> ____________________________________________________________________________________
> Have a burning question?  
> Go to www.Answers.yahoo.com and get answers from real people who know.
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/