Re: Re: Re: st: a user-written program for clustering SE on more than one clustering variable?

[Stas Kolenikov <skolenik@gmail.com>]

Defensibility depends on who you talk to. In my experience, economists
would be happier with the clustered standard errors, as this method
makes fewer assumptions about the data (like the specific structure of
the error terms that -xtmixed- has to assume... although it still
makes some difficult to assess assumptions of uncorrelatedness of e_it
and e_js terms for i!=j, t!=s; violation of this assumption is what
produces the negative variance estimates). For health scientists,
mixed models are far better understood, give smaller standard errors
in small samples (which is what health people apparently have to deal
with more often than economists) and the context of the clustered
standard errors is mostly that of GEE. Education and psychology people
have little clue about clustered standard errors at all, as nearly all
of their models are multilevel models (even the name is different!).

If I were to see drastically different results from -regress- and
-xtmixed-, I would rather want to take -regress- more seriously, as it
makes fewer assumptions. Getting the right standard errors is going to
be a big pain in the lower back, but if I wouldn't get what I needed
from -cmgreg- and the like (once again, -ivreg2- must be able to do
this, as far as I can recall, and is in Stata mainstream, meaning
better coded, better documented and better understood by the user
base), I would look at Art Owen's multiway bootstrap
(http://www.citeulike.org/user/ctacmo/article/11402489), although
again it makes assumptions similar to those underlying -cmgreg-
regarding independence of observations that do not overlap on any
dimensions.

-- Stas Kolenikov, PhD, PStat (ASA, SSC)
-- Senior Survey Statistician, Abt SRBI
-- Opinions stated in this email are mine only, and do not reflect the
position of my employer
-- http://stas.kolenikov.name



On Sat, Aug 10, 2013 at 8:45 AM, Ariel Linden, DrPH
<ariel.linden@gmail.com> wrote:
> Stas,
>
> I agree with your (and Austin's) assessment. In addition, I spent a fair
> chunk of yesterday examining the behavior of -cgmreg- compared to -xtmixed-
> nesting at two and three levels (as well as -xtreg- with re and fe options).
> In almost all cases (with the data I was using), cgmreg gave coefficient
> estimates that were nearly identical to -regress-, with of course, changes
> to the se's. However, these coefficients were much different than those
> derived from -xtmixed- (in fact, in some cases the sign of the coefficient
> went from positive to negative).
>
> Given this, I think my conclusion is to stick with "generally accepted
> practices" regarding multilevel clustered data and use the -xtmixed- or
> -gllamm- (findit gllamm). The approach will be more defensible to reviewers
>
> Ariel
>
> ________________________________________
> From
>   Stas Kolenikov <skolenik@gmail.com>
> To
>   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
> Subject
>   Re: Re: st: a user-written program for clustering SE on more than one
> clustering variable?
> Date
>   Sat, 10 Aug 2013 07:47:58 -0500
> ________________________________________
> As far as I can recall, a couple of years back Austin Nichols and I
> looked into this, and found that -cmgreg- replaces some negative
> values on the diagonal by zeroes. That way, the users does not really
> know (1) when the model assumed by the estimator is wrong, as would
> have been evidenced by a negative variance / missing standard error,
> (2) how many degrees of freedom are left for the estimator once some
> of them are taken out (which is a sore point in clustered standard
> errors). So use at your own risk.
>
> -- Stas Kolenikov, PhD, PStat (ASA, SSC)
> -- Senior Survey Statistician, Abt SRBI
> -- Opinions stated in this email are mine only, and do not reflect the
> position of my employer
> -- http://stas.kolenikov.name
>
>
>
> On Fri, Aug 9, 2013 at 4:36 PM, Ariel Linden, DrPH
> <ariel.linden@gmail.com> wrote:
>> Actually, I finally found it. The program is called -cgmreg- and is found
> at
>> http://www.econ.ucdavis.edu/faculty/dlmiller/statafiles/
>>
>> The approach is discussed in a series of working papers that resulting in
>> the final journal article:
>>
>> Colin Cameron, Jonah Gelbach, and Douglas L Miller, "Robust Inference with
>> Multi-way Clustering", Journal of Business and Economic Statistics, 2011.
>>
>> Thanks
>>
>> Ariel
>>
>
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/