st: Re: Statistical significance vs the 95% confidence intervals -- howshould i interpret these

["Michael Blasnik" <michael.blasnik@verizon.net>]

Neither of your approaches is correct.  In your first approach, you are 
being overly conservative by just checking whether confidence intervals 
overlap (but you aren't very far off).  In your second approach, -ttesti- is 
looking for standard deviations, not standard errors.  Look at the output 
and you will see how ttest has created a new standard error that is much 
smaller (approxiately s.d./sqrt(n)).

Your question would be best answered by using whatever estimation method you 
used to come up with the estimates and standard errors and then use 
Stata's -test- or -lincom- commands to test hypotheses that properly account 
for potential covariance.  If you can't do that for some reason, then you 
would need to make an assumption about the covariance.  If you assume that 
the two estimates are independent, then you can calculate a t-statistic on 
the difference based on the difference in the estimates and the standard 
error of this difference:

di (.484721-.4556235)/sqrt(.015637^2+.0138994^2)
1.3907943

So t=1.39.  You can get a p-value from:

di ttail(2398,(.484721-.4556235)/sqrt(.015637^2+.0138994^2))
.08220843

(or use whatever d.f. are appropriate if 2398 is not, but it won't make any 
real difference here)

So you could conclude that, if the two estimates are independent, then the 
difference between year1 and year3 is not quite statistically significant at 
the .05 level.


Michael Blasnik

----- Original Message ----- 
From: "Columbia & Belmont Apt" <bedindc200@yahoo.com>
To: <statalist@hsphsun2.harvard.edu>
Sent: Friday, December 08, 2006 12:25 PM
Subject: st: Statistical significance vs the 95% confidence intervals -- how 
should i interpret these



> Dear Statalist Colleagues,
> This might be a simple question but I cannot reconcile the following 
> results.
>
> The question is "Are the estimates for year 1 and 3 statistically 
> different?
> year    estimate    st.error        95% conf interval
> year1    .484721     .015637     .454068    .5153741
> year2   .4128893    .0145616    .3843443    .4414343
> year3   .4556235    .0138994    .4283765    .4828704
>
> 1. Using the 95% confidence interval that I obtained:
>
> Year 1 and year 3 confidence intervals overlap thus year 1&3 estimates are 
> not statistically different at the 95% level.
>
> 2. Using ttesti they are statistically different with 99% confidence:
>
> . ttesti 2398 .484721  .015637 2399 .4556235 .0138994
> Two-sample t test with equal variances
> ------------------------------------------------------------------------------
>         |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. 
> Interval]
> ---------+--------------------------------------------------------------------
>       x |    2398     .484721    .0003193     .015637    .4840948 
> .4853472
>       y |    2399    .4556235    .0002838    .0138994     .455067 
> .45618
> ---------+--------------------------------------------------------------------
> combined |    4797    .4701692    .0002996    .0207488    .4695819 
> .4707565
> ---------+--------------------------------------------------------------------
>    diff |            .0290975    .0004272                  .02826 
> .029935
> ------------------------------------------------------------------------------
> Degrees of freedom: 4795
>                      Ho: mean(x) - mean(y) = diff = 0
>     Ha: diff < 0               Ha: diff != 0              Ha: diff > 0
>       t =  68.1143                t =  68.1143              t =  68.1143
>   P < t =   1.0000          P > |t| =   0.0000          P > t =   0.0000
>
>
> Which one is the correct? What am I not interpreting right?
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/