st: numerical derivatives and -ml- command

[Sun Yutao <yutao.sun.statalist@outlook.com>]

Hello,

I'm trying to write an maximum likelihood algorithm that can handle a 
particularly large number of fixed effects (which is not possible with Stata's 
#var and matsize limitation). I have been comparing the performance of my ml 
with the Stata -ml- these days and I find something really strange. More 
specifically, does anyone know how the Newton-Raphson algorithm in Stata -ml- 
command works in detail? Because what I see from a small experiment is that 
the -ml- command is just way too fast so it's not possible to iteratively 
update the Hessian by numerical differentiation.

Here the story goes:

For an original Newton-Raphson(nr), one needs to update the gradients and 
Hessian in each iteration.  And suppose we have a likelihood function(llf), 5 
variables, and a given number of observations. And a single evaluation of the 
llf takes 0.0475 seconds, so for a 2-point approximation of the gradient 
vector one need 5*2=10 evaluations, which gives roughly 0.475 seconds. And the 
Hessian is a bit complicated: every off-diagonal element will need 4 llf 
evaluations and you have  4+3+2+1=10 of them, while the diagonal elements need 
2 evaluations each, plus 1 for the common f(x), which gives 5*2+1 llf 
evaluations and hence for the Hessian one needs 10*4+5*2+1=51 llf evaluations 
which gives  2.4225 seconds.

Based on that, one iterations in this setting should roughly take 3 seconds. 
However, the Stata -ml- command only needs 1 second per iteration, which is 
particularly uncommon. And even if my calculations for the Hessian was wrong, 
I'm still convinced that the Hessian needs more llf evaluations and hence more 
time to compute than the gradients. So it cannot possibly be 1 second per 
iteration. And in fact, in terms of performances, the Stata nr behaves a 
little bit like a quasi-Newton + a line search... Or is It possible that Stata 
secretly have a feature that can take analytical derivatives on a 
user-specified function?

Does anyone know the answer to this problem? Or if there just is a better way 
to compute numerical derivatives that I do not know?

Best regards,
Sun Yutao



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/