st: Clustered Standard Errors vs HLM for Small Sample Project

[<S.Jenkins@lse.ac.uk>]

As long as you really don't want to say anything about the level-2 (state) predictors, and only the level-1 (agency) ones, you may be in luck ... because some Monte-Carlo analysis suggests that the estimates and SEs for the level-1 variables may be OK even if the number of level-1 obs per cluster is very "small", as in your case.  Observe also that the nice properties of cluster-robust SEs also requires a "large" number of clusters, other things being equal.

For a review of approaches, and citations to related literature, see 
'Regression analysis of cross-national differences using multi-level data: a cautionary tale', Working Paper 2013-14. Colchester: Institute for Social and Economic Research, University of Essex. https://www.iser.essex.ac.uk/publications/working-papers/iser/2013-14 . (Or get it as IZA Discussion Paper No. 7491, http://ftp.iza.org/dp7491.pdf.)
Some research suggests that Bayesian approaches (MCMC estimators and all that) may do better than standard ones (e.g. adaptive quadrature as in Stata).
Also relevant is whether your outcome variable is continuous or discrete/categorical. (Estimators are more likely to have poor properties in non-linear models than linear ones.) 


Stephen
------------------
Stephen P. Jenkins <s.jenkins@lse.ac.uk>

------------------------------

Date: Mon, 18 Nov 2013 02:06:01 +0000 (UTC)
From: mkobren1@comcast.net
Subject: st: Clustered Standard Errors vs HLM for Small Sample Project

I'm using STATA 10 and I'm trying to figure out whether to use clustered standard errors or HLM.I have 233 observations from agencies located in 10 different states. 

The minimum number of observations I have from a state is 3 and the maximum number of observations I have is 108 with an average 
of 23.3. I'm not interested in state level differences, I'm only interested in results from the agency level and I want to account for the fact that there may be some state level effects.

The literature I've read so far doesn't seem to point me in any definite direction. The literature seems to say that HLM works best on larger datasets, but it also seems to say that you need at least 20 clusters for either method to be effective. Does anyone have a suggestion for which of these two methods I should use, or at least what I should consider in making my choice? Is there some other method I should use?

Thank you in advance for your consideration.

MK

Please access the attached hyperlink for an important electronic communications disclaimer: http://lse.ac.uk/emailDisclaimer

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/