Re: st: test or ftest command

[Lisa Marie Yarnell <lisayarnell@yahoo.com>]

Hi everyone,

Hopefully this is not a silly question, but how do I test for the significance of the random effect in the model below?  

gllamm w3bmi_ms bio_sex w3_yrs w1faminc black hispanic mothed selfed sup_dens ffchdens nat_dens con_dens veg_dens act_dens med_hval, i(scid) family(gaussian) link(identity) eform pweight(wt)

Is testing the for the significance of the random effect the same as testing for the significance of (or support for) clustering subjects within schools (scid)?

I'm working on a project where we didn't cluster persons within schools, but a reviewer commented that perhaps we should have.  So we're trying to conduct a basic test of whether clustering is important.  Another Statalister had responded below.  Many thanks for everyone's help.

Lisa






--- On Wed, 6/8/11, Joerg Luedicke <joerg.luedicke@gmail.com> wrote:

> From: Joerg Luedicke <joerg.luedicke@gmail.com>
> Subject: Re: st: test or ftest command
> To: statalist@hsphsun2.harvard.edu
> Date: Wednesday, June 8, 2011, 2:41 PM
> On Wed, Jun 8, 2011 at 1:28 PM, Lisa
> Marie Yarnell
> <lisayarnell@yahoo.com>
> wrote:
> > Hello Stata group!  I am trying to do an ftest (or
> other test possibly using the "test" command) comparing two
> models: one regression model in which clustering of persons
> within schools is modeled, and one regression with
> clustering of persons within schools *not* modeled.
> >
> > I am concerned that these are not really nested
> models, however.  Can I still use the test or ftest
> command?  Is there another command that I should be using?
>  Our basic question is: "Is it important to model the
> clustering of persons wihtin schools, based on comparisons
> of model fit?"  Here are the models:
> >
> > (1) regress w3bmi_ms bio_sex w3_yrs w1faminc black
> hispanic mothed selfed sup_dens ffchdens nat_dens con_dens
> veg_dens act_dens med_hval, cluster (scid)
> >
> > (2) regress w3bmi_ms bio_sex w3_yrs w1faminc black
> hispanic mothed selfed sup_dens ffchdens nat_dens con_dens
> veg_dens act_dens med_hval
> >
> > Can I use the test command for this?  When I tried, I
> got the error message below; Stata treated "cluster(scid)"
> as a variable in the model:
> >
> > regress w3bmi_ms bio_sex w3_yrs w1faminc black
> hispanic mothed selfed sup_dens ffchdens nat_dens con_dens
> veg_dens act_dens med_hval, cluster (scid)
> >
> > test cluster(scid)
> > cluster not found
> >
> 
> The above two models only differ in the way the standard
> errors are
> calculated. The models itself are the same and, thus, there
> is no
> difference in model fit. If I understand your problem
> correctly ("Is
> it important to model the clustering of persons within
> schools"), this
> boils down to the question to what extent you observe any
> intra-class
> correlation. With students nested in schools this is
> usually the case
> (I guess) and the standard way to handle these type of data
> in
> educational research is a multilevel model. However, to
> start, I would
> suggest looking at possible intra-school correlation. For
> example you
> could try:
> 
>  xtreg w3bmi_ms bio_sex w3_yrs w1faminc black hispanic
> mothed selfed
> sup_dens ffchdens nat_dens con_dens veg_dens act_dens
> med_hval,
> i(scid)
> 
> "rho" can then be interpreted as a measure of intra-class
> correlation.
> 
> You could also use a maximum-likelihood estimator in which
> case Stata
> returns a likelihood ratio test where the model with the
> random effect
> is compared to the model without random effect:
> 
> xtreg w3bmi_ms bio_sex w3_yrs w1faminc black hispanic
> mothed selfed
> sup_dens ffchdens nat_dens con_dens veg_dens act_dens
> med_hval,
> i(scid) mle
> 
> This is the same as a mixed model using xtmixed with
> varying
> intercepts across schools and maximum-likelihood
> estimation:
> 
> xtmixed w3bmi_ms bio_sex w3_yrs w1faminc black hispanic
> mothed selfed
> sup_dens ffchdens nat_dens con_dens veg_dens act_dens
> med_hval ||
> scid:, mle
> 
> 
> HTH,
> 
> J.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/