Re: st: Comparison of robust and cluster-robust standard errors when the number of clusters is small

["Tobias Pfaff" <tobias.pfaff@uni-muenster.de>]

Dear Austin,

Thanks, Rogers (1993) was a good hint!

I understand that the intracluster correlation (rho) can be estimated with
-loneway-.
However, Feng et al. (2001) say that the estimation of the intraclass
correlation coefficient is poor if the number of groups is small. The bias
of rho is downward. And indeed, -loneway- returns a rho of zero in my case.

Roger (1993) says that the reason for the downward bias of the
cluster-robust SE in the case of few clusters is "that there are
mathematical constraints on the residuals." My stats background is not good
enough to tell if the reason for the downward bias of the ICC coefficient
(rho) in the case of few clusters is just the same.

Without reliable estimation of rho I just find it hard to say if negative
correlation within cluster adds to the phenomenon that my cluster-robust SE
is smaller than the robust SE. 

Regards,
Tobias

Literature cited:
Feng, Z., Diehr, P., Peterson, A., & McLerran, D. (2001). Selected
statistical issues in group randomized trials. Annual Review of Public
Health, 22, 167-187.



On Fri, Jan 4, 2013 at 10:35 AM, Austin Nichols
<austinnichols@gmail.com> wrote:

> As the FAQ says,
> "See the manual entries [R] regress (back of Methods and Formulas),
> [P] _robust (the beginning of the entry), and [SVY] variance
> estimation for more details."
> and see an article referenced there (and by
> http://repec.org/usug2007/crse.pdf
> as well):
> http://www.stata.com/support/faqs/stat/stb13_rogers.pdf
> --Rogers 1993 is still the best intro, though the commands are all
obsolete.
> Also see Kish 1965
> http://www.amazon.com/Survey-Sampling-Wiley-Classics-Library/dp/0471109495
> for an early explanation of one way to measure [possibly negative]
> intracluster correlation of x*e, using a quantity which Kish calls roh
> [sic].

On Thu, Jan 3, 2013 at 11:43 AM, Tobias Pfaff
<tobias.pfaff@uni-muenster.de> wrote:
> Hi all,
>
> The Stata FAQ explains nicely why cluster-robust standard errors
> (-vce(cluster clustvar)-) can be smaller than robust standard errors
> (-vce(robust)-):
>
http://www.stata.com/support/faqs/statistics/standard-errors-and-vce-cluster
> -option/
>
> The FAQ's answer is negative correlation within cluster.
> But could it be that in cases with small number of clusters this answer is
> not sufficient?
>
> Consider a setting with a small number of clusters (in my case 12
clusters)
> and the following standard errors:
>
> Ordinary (OLS) SE: .1109
> Robust SE: .1268
> Cluster-robust SE: .0414
>
> The literature says that an insufficient number of clusters (approximately
> less than 50) can lead to standard errors that are downward biased (e.g.,
> Cameron et al. 2008).
>
> Is it correct to say that my cluster-robust SE is smaller than the robust
SE
> due to negative correlation within cluster OR due to downward bias of the
> cluster-robust SE in the case with few clusters?
>
> If the statement is correct, can I find out if one of the reasons can be
> ruled out? Can I measure the negative correlation? Or can I measure the
> downward bias due to few clusters?
>
> In this regard, maybe you guys have a hint for me why (mathematically) the
> SE are downward biased in the case with few clusters? I didn't find an
> answer so far in the literature.
>
> Any comments are appreciated!
>
> Thanks very much,
> Tobias
>
> Literature cited:
> Cameron, Gelbach, Miller (2008), Bootstrap-Based Improvements for
Inference
> with Clustered Errors. The Review of Economics and Statistics, 90 (3),
> 414-427.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/