Re: st: mata optimize with d2debug

[Partha Deb <partha.deb@hunter.cuny.edu>]

Stas - although I have no great insight, I wonder if the numerical 
derivatives are even meaningful along the ridge.  In other words, one 
might expect mreldif to be big because the numerical derivatives are 
*not* good approximations to the analytical gradients on the ridge.

In coding up -fmm- , I discovered this feature when the parameters in 
each component were set to be the same - in that case, it's like being 
on a ridge because the mixing probabilities are not identified.  Stata's 
numerical derivatives are, then, quite different from the analytical 
ones.  But off that "ridge", the analytical and numerical derivatives 
are very close.

So, my gut instinct is that, you should not worry about the 
discrepancies quite so much.

cheers.

Partha


Stas Kolenikov wrote:
> I am programming a fairly large model and rather poorly identified
> model with a couple dozen parameters in Mata. Documentation on
> -mf_optimize- says: "When you have done things right, gradient vectors
> will differ by approximately 1e–12 or less and Hessians will differ by
> 1e–7 or less." I never get there; even a restricted version of the
> model that is known to converge well produces mreldifs of about 1e-7
> and 1e-4, respectively. The mreldifs for the Hessian might start kinda
> high between 1 and 10, but they would eventually go down near the
> maximum. For the interesting (and poorly identified) models that I
> eventually want to fit, I get mreldifs around 1e-3 to 1e-5 in
> gradients for most iterations, while my mreldifs for the Hessian
> fluctuate between 1e-3 and 1. In the early steps far from the maximum,
> the mreldifs for the Hessian can be as large as 100 or so (that's for
> 20x20 matrix, remember), but they go down as I converge to the
> maximum. Since I am walking along a ridge, I would actually tend to
> trust my analytical derivatives more than I do the numeric
> derivatives. Is that reasonable? Any advice on this? I tried tighter
> convergence criteria, but the results did not change much.
>
>   

--
Partha Deb
Professor of Economics
Hunter College
ph:  (212) 772-5435
fax: (212) 772-5398
http://urban.hunter.cuny.edu/~deb/

Emancipate yourselves from mental slavery
None but ourselves can free our minds.
	- Bob Marley


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/