 |
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: finite mixture models with the EM algorithm
László,
Stas is correct in that EM would have to be written from scratch and
implemented case by case.
In my experience, using models with covariates and sample sizes in the
thousands generally, EM has been *much* slower than ML, in part because
of the number of M-steps needed for convergence. Also, while EM might
be able to deal with multimodalities better than ML (esp. Stata's -ml-),
there is nothing inherent about EM that makes it "better". Perhaps one
ought to try simulated annealing if one is really worried. In my
experience, messing around with the starting values has been
sufficiently comforting.
Although the convergence problems you are describing are well within the
realm of possibilities of an otherwise well specified finite mixture
model, in my experience, such severe convergence issues are almost
always indication of an otherwise misspecified model (or a poorly
specified model vis-a-vis the assumed mixture). I've observed that
convergence is more difficult when one has indicator variables with low
frequencies, especially when the mixing probabilities are "lopsided" -
perhaps this is an issue. What happens if you estimate a sparsely
specified model? What does the kernel density of the outcome look like?
cheers,
Partha
László Sándor wrote:
Thanks very much, Stas!
I just heard today in lecture from Guido Imbens that EM might solve
the problem when the likelihood is multimodal, as in the case of
mixtures. Can we reasonably expect Stata's ML procedure to solve this
problem anyway?
With -fmm-, convergence seemed to be really tricky for me. I had to
dig deep and enable -ml search- and set a large number of repetitions
for it. Plus I have the difficult option enabled. Even then, sometimes
I get no convergence just telling me that the likelihood is concave,
or some of the models converged to a case with basically only one
component present out of the possible two -- which is not impossible,
but unlikely in my case.
Thanks again,
Laszlo
On Thu, Sep 17, 2009 at 2:49 PM, Stas Kolenikov <[email protected]> wrote:
The EM algorithm might converge faster, that's all. You will get the
same estimates, although without the standard errors; they are the
pain in the lower back with the EM. If -fmm- works for you, there is
little reason to try and code the EM algorithm from scratch, unless
you'd be lucky to find an implementation floating around. Unlike the
general -ml-, the EM algorithm is problem-specific and depends on what
your missing data are. In -xtmixed-, the missing data are continuous
random effects, while in mixture models, the missing data are discrete
class indicators. The E-steps are entirely different in those two
problems.
2009/9/17 László Sándor <[email protected]>:
Dear fellow statalisters,
I am estimating a model where I assume that the dependent variable
came from either of two classical normal linear models, with some
probabilities for the one and the other. This is a finite mixture
model, and I had some success with Partha Deb's -fmm- command.
However, I heard it might be very much worthwhile to try the
estimation using the EM algorithm, not standard (though fine-tuned á
la Stata) ML. Before any more Google searches getting lost among
latent class models and various related topics, I would ask you
whether you happen to know a tool (or way) that can do this estimation
using the EM algorithm.
(I know that there are different software packages claiming to do
this, but I would rather stick to Stata. Especially because Stata does
EM for -xtmixed-, for instance, so maybe it is also easily applicable
to my problem somehow.)
Thank you very much, you're help is very much appreciated, as always.
Laszlo
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
--
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
--
Partha Deb
Professor of Economics
Hunter College
ph: (212) 772-5435
fax: (212) 772-5398
http://urban.hunter.cuny.edu/~deb/
Emancipate yourselves from mental slavery
None but ourselves can free our minds.
- Bob Marley
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/