Jean-Benoit Hardouin wrote:
I just figured I'd offer some alternative perspective on Jean-Benoit's
very informative comments.
>>I think your problem of convergence is a consequence of your small
sample size. You have only N=40 subjects for J=30 items. Generally in
IRT, we consider to be in good conditions if J<<N. In order to achieve
this requierement, sample size are generally important (2000-30000
subjects) if the test (questionnaire) is long (typically in Educational
Sciences) or if the sample size is small (100-300 individuals), the test
is short (J=5-10) (typically in Health Sciences).<<
True enough but I guess I'd say that you can relax this somewhat
provided you know what you're doing. Certainly Stas qualifies as "knows
what he's doing."
>>I think your sample is too small to envisage a complex IRT models like
the 2 parameters logictic model (2PLM or Birnbaum model) (60
parameters=30 discriminating powers (factor loadings) minus 1
(identifiability constraint), 30 difficulty parameters (fixed effects),
and the variance of the latent variable (which generally is not fixed to
one). Even for the Rasch model (1PLM) which consider only 31 parameters
(30 difficulty parameters and the variance of the latent variable), your
sample is small !!<<
This is where Bayesian estimation (deterministic or stochastic) can be
VERY helpful. You can fit a model that's a compromise between the Rasch
and 2PL by using a hyper-parameter on the slopes, for instance, to
shrink things towards a common mean value. Make this prior very
informative and you have a Rasch model. Make it very uninformative and
you have a 2PL model.
>>For most of psychometricians, the Rasch model (and its polytomous
extensions like the Rating scale model or the Partial Credit Model) is
the only one (IRT) model which allows obtaining an objective measure (a
measure independent of the sample, and independent of the responded
items), so the others IRT models are not recommanded.<<
Just to note this is an area of substantial dispute. The 2PL model is
the Spearman factor model analog for logistic regression. If you like
the Spearman factor model but hate the 2PL, there's a conflict in
reasoning.
>>Generally, we
don't obtain a better measure with a complex IRT model than by using the
classical score computed as the number of correct responses. A complex
IRT model can only be a way to understand the items functionning (is a
guessing effect, a strong discrimination power...). So I always
recommand to use the Rasch model in a first intention.<<
Agreed. If you're *making* a test, use the Rasch model if at all
possible. The problem with it is the fact that often we don't get to
pick the dataset we're analyzing. When you fit a Rasch model to data
from a different population, it can do some decidedly odd things.
>>Concerning the fit of a 2PLM, if you have SAS, you can easily test the
convergence of the estimations by using the %anaqol macro-program
(available on http:\\www.anaqol.org). This macro use the NLMIXED
procedure which is based on the same technics of estimation than
-gllamm-, so these two procedure are comparable (even on the computing
time, usually very long !!!!!).<<
NLMIXED has one really big advantage over -gllamm- in many
circumstances. It uses analytic derivatives via automatic
differentiation rather than numerical derivatives. This is a potentially
huge speedup because it cuts the number of fevals down a lot. I have
gotten quite complex models to run in NLMIXED quite rapidly (seconds to
minutes) given good starting values. Sure, it's not as quick as, say,
BILOG, but it's a whole lot more flexible. It is imperative that it have
good starting values, however.
One of my mentors, Carolyn Anderson, has done some nice work on
pseudo-likelihood estimation of very large IRT models. She's got a more
technical article somewhere in the Psychometrika pipeline, but for a
simpler version see:
Anderson, C. J., Li, Z., & Vermunt, J. K. (2007). Estimation of models
in a Rasch family for polytomous items and multiple latent variables.
Journal of Statistical Software, 20(6), 1-36.
All of these models could be ported in Stata quite easily.
JV
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
