Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Very high t- statistics and very small standard errors
From
Richard Williams <[email protected]>
To
[email protected], [email protected]
Subject
Re: st: Very high t- statistics and very small standard errors
Date
Tue, 01 May 2012 09:10:19 -0500
At 04:17 AM 5/1/2012, Maarten Buis wrote:
On Tue, May 1, 2012 at 3:18 AM, Laurie Molina wrote:
> It is not the first time I hear people say that when you have a lot of
> observations everything is significant... Is it because the lenght of
> the confidence intervals is inversely related to the number of
> observations considered? Or could you tell me what is the logic behind
> saying that with a lot of observations everything is statistically
> significant?
The logic is that statistical testing is all about the random
variation in your coefficients you would expect due to the fact your
data is a random sample of the population. You would expect that if
you draw a sample you would not find exactly the same statistic as you
would expect under the null hypothesis even if the null hypothesis is
true. Statistical testing is all about the probability that the
estimate you found could have been drawn "by accident" if the null
hypothesis is true. When a statistic is unlikely to have been the
result of such an "accident" we call it significant. In small samples
you could more easily be "unlucky" and draw a "weird" sample with very
different coefficients than the population. Such accidents are a lot
less likely when you draw large samples than small samples, so in
large samples we should get more significant results.
It may also be the case that the null hypothesis is not exactly true,
e.g. the difference between men and women isn't $0, it is $2. Such
substantively trivial differences will show up as insignificant in a
small sample but significant in a large one.
I suspect too that even the most spectacular and gigantic sample will
have minor flaws (e.g. nonresponse bias) that will bias its estimates
a little bit. So, even if the difference really was 0, the sample
estimate wouldn't be that.
-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
OFFICE: (574)631-6668, (574)631-6463
HOME: (574)289-5227
EMAIL: [email protected]
WWW: http://www.nd.edu/~rwilliam
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/