Initial reaction: Your model is either unidentified or very weakly identified and you are essentially at a "crease" in the log-likelihood manifold with a very slight peak. The different answers happen as you explore different regions of the crease. This is just a guess though. Are their predicted values similar?
You may want to try smoothing a little. An easy way to do this is to add X observations that are uniformly distributed from the option set (or just distributed as per the univariate marginal for the options). Make X a relatively small proportion of the dataset, say 1% to 5% and be sure to move the number around to make sure the result is stable. This will shrink everything towards non-significance but that's the price you have to pay.
I could be off-base so I look forward to hearing others' suggestions.
-----Original Message-----
From: "KONSTANTARAS KONSTANTINOS" <[email protected]>
To: [email protected]
Sent: 11/24/2008 9:39 AM
Subject: st: GLLAMM multinomial: tremendous instability
Dear statalisters,
In my model runs (using Intercooled Stata 9.2) I experience tremendous instability in gllamm results, higher than has been elsewhere reported at statalist: I run a simple multinomial logit model with random effects, using the results as initial matrix for exactly the same model, to get as a result an extremely different log likelihood and coefficients etc. The resulting loglikelihood differs by large margins (3-9%). I have also tried to use exactly the same initial values from mlogit but not only does the overall loglikelihood continues to be unstable but also the same happens with the loglikelihood obtained without random effects –running gllamm with init option-. Nevertheless the model invariably converges, albeit to an entirely different estimate.
I have witnessed this instability in my initial model consisting of 8 covariates and three outcomes, hence, following the typical in relative cases recommendation from statalist to start “small” I have only used 1 explanatory variable, but instability remains –see for example the output below-.
This problem, the way I read it, does not lie on the data set, as I get stable results from either mlogit -without random effects- or asmprobit -with random effects- converging steadily and nicely irrespective of starting matrix. By the way, asmprobit indicates existence of unobserved heterogeneity. Hence it’s not a multimodal maximum likelihood problem, or sparseness, because it would show up more intensely in MSL methods.
Then what might the problem be? Any suggestions on how to go about it -because I would like to use the discrete Latent class unobserved heterogeneity capability of gllamm-?
Thanks in advance for your time taken to assist me!
Dino Konstantaras
PS.
I use panel data consisting of 390 highly unbalanced observations with gaps from 129 -level 1- units (id). Outliers (as detected by hadimvo routine) have been removed. My dependent categorical variable assumes 3 distinct values 1-3, 3 being the base reference with the highest frequency. My covariates consist of two binary and six continuous covariates –standardized- . The example below has been run with one continuous standardized covariate. The lower EPV (events per variable) in the run below is 30.
* GLLAMM output *
. gllamm alt fmt0 , expand(chosen patt m) l(mlogit) f(bin
> om) adapt i(firm) nip(8) base(3) nrf(2) eqs(a1 a2)
Running adaptive quadrature
Iteration 0: log likelihood = -297.50547
…
Iteration 5: log likelihood = -295.55745
Adaptive quadrature has converged, running Newton-Raphson
Iteration 0: log likelihood = -295.55745
Iteration 1: log likelihood = -295.55744
Iteration 2: log likelihood = -295.55743
number of level 1 units = 1170
number of level 2 units = 129
Condition Number = 18.400152
gllamm model
log likelihood = -295.55743
------------------------------------------------------------------------------
alt | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
c1 |
fmt0 | 1.785588 2.034928 0.88 0.380 -2.202797 5.773973
_cons | .3614228 .2573106 1.40 0.160 -.1428968 .8657423
-------------+----------------------------------------------------------------
c2 |
fmt0 | 3.625611 1.826175 1.99 0.047 .0463731 7.204849
_cons | .3541614 .2666299 1.33 0.184 -.1684235 .8767463
------------------------------------------------------------------------------
Variances and covariances of random effects
------------------------------------------------------------------------------
***level 2 (firm)
var(1): .12708114 (.66898288)
cov(2,1): .16044789 (.83246548) cor(2,1): 1
var(2): .2025755 (1.0658552)
------------------------------------------------------------------------------
. mat a0=e(b)
. gllamm alt fmt0, expand(chosen patt m) l(mlogit) f(bin
> om) adapt i(firm) nip(8) base(3) nrf(2) eqs(a1 a2) from(a0)
Running adaptive quadrature
Iteration 0: log likelihood = -293.33182
…
Iteration 3: log likelihood = -288.31959
Adaptive quadrature has converged, running Newton-Raphson
Iteration 0: log likelihood = -288.31959
Iteration 1: log likelihood = -288.31959
Iteration 2: log likelihood = -288.31958
number of level 1 units = 1170
number of level 2 units = 129
Condition Number = 17.167532
gllamm model
log likelihood = -288.31958
------------------------------------------------------------------------------
alt | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
c1 |
fmt0 | 5.718242 1.914655 2.99 0.003 1.965587 9.470898
_cons | .585764 .2601162 2.25 0.024 .0759457
[truncated by sender]
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
