Maximum likelihood estimation

In addition to providing built-in commands to fit many standard maximum likelihood models, such as logistic, Cox, Poisson, etc., Stata can maximize user-specified likelihood functions. To demonstrate, imagine Stata could not fit logistic regression models. The logistic likelihood function is

\(f(y, Xb) = 1/(1+exp(-Xb)) \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\) if \(y = 1\)
\(\;\;\;\;\;\;\;\;\;\;\;\;\: = exp(-Xb)/(1+exp(-Xb)) \;\;\;\;\;\;\;\;\;\;\) if \(y = 0\)

We might first write a program in Stata to calculate the log of the likelihood function given \(y\) ($ML_y1 in the code below) and \(Xb\):

. program define mylogit
          args lnf Xb
          quietly replace `lnf' = -ln(1+exp(-`Xb')) if $ML_y1==1
          quietly replace `lnf' = -`Xb' - ln(1+exp(-`Xb')) if $ML_y1==0
  end

That done, we can fit a logistic-regression model of dependent variable foreign on mpg and displ by typing

  . ml model lf mylogit (foreign=mpg weight)
  . ml maximize

You will be surprised when you see the output:

 . ml model lf mylogit (foreign=mpg weight)

 . ml maximize

 Initial:       Log likelihood = -51.292891
 Alternative:   Log likelihood = -46.081697
 Rescale:       Log likelihood = -45.181365
 Iteration 0:   Log likelihood = -45.181365
 Iteration 1:   Log likelihood = -29.420506
 Iteration 2:   Log likelihood = -27.210884
 Iteration 3:   Log likelihood = -27.175196
 Iteration 4:   Log likelihood = -27.175156
 Iteration 5:   Log likelihood = -27.175156

                                                         Number of obs =     74
                                                         Wald chi2(2)  =  17.78
Log likelihood = -27.175156                              Prob > chi2   = 0.0001





      foreign 
 
 Coefficient  Std. err.      z    P>|z|     [95% conf. interval]



          mpg 
 
  -.1685869   .0919175    -1.83   0.067    -.3487418     .011568

       weight 
 
  -.0039067   .0010116    -3.86   0.000    -.0058894    -.001924

        _cons 
 
   13.70837   4.518709     3.03   0.002     4.851859    22.56487

Stata automatically generated this neatly formatted output, complete with significance levels and confidence intervals.

The following features are worth noting:

• We define a program to calculate the log likelihood generically—in terms of \(y\) and \(Xb\).
• Once the program to calculate the log likelihood has been defined, we can fit any particular model. The syntax of the ml model statement is

  ml model methodname programname ( model )

  To fit a model of foreign on mpg and weight using our program mylogit (and method lf), we typed

  ml model lf mylogit (foreign=mpg weight)

• We do not have to specify initial values for the parameters, although we may. When we do not specify initial values, ml finds starting values on its own.
• Stata provides four optimization methods: lf, d0, d1, and d2. Optimization method lf is easiest to use because (1) you do not have to program derivatives and (2) the likelihood is assumed to be of the form \(L(Xb)\). Method d0 allows the function to be more general, \(L(X,b)\), but similarly requires no programming of derivatives. Method d1 requires programming first derivatives, and method d2 requires programming first and second derivatives.

Stata’s likelihood-maximization procedures have been designed for both quick-and-dirty work and writing prepackaged estimation routines that obtain results quickly and robustly. For instance, Stata fits negative binomial regressions (a variation on Poisson regression) and Heckman selection models. We wrote those routines using Stata's ml command, although most users are not aware of that. They think that negative binomial and Heckman selection are just two more things Stata can do.

If you are serious about maximizing likelihood functions, you will want to obtain the text Maximum Likelihood Estimation with Stata, Fifth Edition by Jeffrey Pitblado, Brian Poi, and William Gould (2024). The first chapter provides a general overview of maximum likelihood estimation theory and numerical optimization methods, with an emphasis on the practical applications of each for applied work. The middle chapters detail, step by step, the use of Stata to maximize community-contributed likelihood functions. The final chapters explain, for those interested, how to add new estimation commands to Stata.