Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Very high t- statistics and very small standard errors
From
Rob Ploutz-Snyder <[email protected]>
To
[email protected]
Subject
Re: st: Very high t- statistics and very small standard errors
Date
Tue, 1 May 2012 08:55:14 -0500
Laurie,
I'll chime in here too, piggy backing on Richard's comment.
Perhaps a more interesting set of significance tests would be to
compare those Beta's to something meaningful, rather than to the
default of zero. It forces you to think about how much of a
difference is really meaningful, but of course, that's the right thing
to be thinking about. And obviously this is something that should be
done before the actual analysis, though I suspect that you already
have the benefit of hind sight...
Rob
On Tue, May 1, 2012 at 9:10 AM, Richard Williams
<[email protected]> wrote:
> At 04:17 AM 5/1/2012, Maarten Buis wrote:
>>
>> On Tue, May 1, 2012 at 3:18 AM, Laurie Molina wrote:
>> > It is not the first time I hear people say that when you have a lot of
>> > observations everything is significant... Is it because the lenght of
>> > the confidence intervals is inversely related to the number of
>> > observations considered? Or could you tell me what is the logic behind
>> > saying that with a lot of observations everything is statistically
>> > significant?
>>
>> The logic is that statistical testing is all about the random
>> variation in your coefficients you would expect due to the fact your
>> data is a random sample of the population. You would expect that if
>> you draw a sample you would not find exactly the same statistic as you
>> would expect under the null hypothesis even if the null hypothesis is
>> true. Statistical testing is all about the probability that the
>> estimate you found could have been drawn "by accident" if the null
>> hypothesis is true. When a statistic is unlikely to have been the
>> result of such an "accident" we call it significant. In small samples
>> you could more easily be "unlucky" and draw a "weird" sample with very
>> different coefficients than the population. Such accidents are a lot
>> less likely when you draw large samples than small samples, so in
>> large samples we should get more significant results.
>
>
> It may also be the case that the null hypothesis is not exactly true, e.g.
> the difference between men and women isn't $0, it is $2. Such substantively
> trivial differences will show up as insignificant in a small sample but
> significant in a large one.
>
> I suspect too that even the most spectacular and gigantic sample will have
> minor flaws (e.g. nonresponse bias) that will bias its estimates a little
> bit. So, even if the difference really was 0, the sample estimate wouldn't
> be that.
>
>
>
> -------------------------------------------
> Richard Williams, Notre Dame Dept of Sociology
> OFFICE: (574)631-6668, (574)631-6463
> HOME: (574)289-5227
> EMAIL: [email protected]
> WWW: http://www.nd.edu/~rwilliam
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/