R: st: change the format of the pseudo log-likelihood in gb2fit

["Lucia R.Latino" <Latino@economia.uniroma2.it>]

Dear Nick,

I am sorry if I was not clear in my previous email. I will try to explain
here better. 

I need to choose the best fit for my data on consumption and I am using
survey data.

I have been using the programs written by Stephen Jenkins (e.g. gb2fit,
smfit, dagumfit, lognfit) using the option svy.

Stata shows the log pseudolikelihood with the scientific notation and it
does not store the loglikelihood in e(ll).

I used the code kindly suggested from Michal Brzezinski to recover the log
pseudolikelihood for the gb2fit and I tried your suggestion of using
-display- with a different format on r(sum). It worked.

In order to get the full likelihood also for the other distributions (e.g.
lognormal, Singh-Maddala, Dagum, Fisk, beta2) following the procedure above,
I would need, as you said, to write a code for the log of each distribution
functions. It's a little bit complicated (for me, at least). 

So, I was wondering if:
1.  there is a way to change the format of the output of -lognfit-, -smfit-,
-dagumfit-, -fisk-, -beta2- in such way that the iterations of the log
pseudolikelihood  are shown not in scientific notation; or
2. there is an easier way to recover the log pseudolikelihood  and store it
after I run each command to fit the distribution; or
3. there is someone who can help me writing the code for the log of each
distribution functions (which I understand it may be just asking too much),
similarly to what Michal Brzezinski did for the gb2.

Best,
Lucia





-----Messaggio originale-----
Da: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Nick Cox
Inviato: venerdì 6 luglio 2012 20:19
A: statalist@hsphsun2.harvard.edu
Oggetto: Re: st: change the format of the pseudo log-likelihood in gb2fit

The last bit involves taking logarithms of some slightly messy expressions.
That's all.

On your first point, I don't understand what you are missing. If the final
log-likelihoods for different models are the same to 4 significant figures,
I suggest that they are to be regarded as identical in practice.
Alternatively, I guess that least one of the following statements is true.

1. You didn't try my suggestion of using -display- with a different format
on r(sum).

2. You did try it, but ran into some problem, in which case you need to
spell out what that problem is.

3. You want something else, but again you need to spell out what that is.
That said, it seems very unlikely that anyone will rewrite these programs
for you.

Nick

On Fri, Jul 6, 2012 at 6:59 PM, Lucia R.Latino <Latino@economia.uniroma2.it>
wrote:
> Dear Nick,
>
> thanks for your hint.
>
> I am trying to compare different nested distributions and  I would 
> love to compare the log likelihood. With the scientific notation I 
> cannot see any difference. That is the reason why I was trying to 
> obtain the full log likelihood.
>
> Combine Michael's code and -display- I am able to get the full log 
> likelihood. But I haven't been able to change how stata display the 
> result after gb2fit or similar command.
>
> Does anyone can help me in writing the explicit expression for 
> log-likelihood for lognfit, dagumfit, smfit, fisk and Beta2 similar to 
> what Michal wrote for the gb2fit?
>
> Thanks,
> Lucia
>
> -----Messaggio originale-----
> Da: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Nick Cox
> Inviato: venerdì 6 luglio 2012 11:58
> A: statalist@hsphsun2.harvard.edu
> Oggetto: Re: st: change the format of the pseudo log-likelihood in 
> gb2fit
>
> -gb2fit- is a user-written program from SSC written by Stephen Jenkins.
>
> -display- takes a format which can be used to change how a result is 
> displayed. Look at the help for -display- and for -format-. However, 
> it is difficult to see that further decimal places could be of either 
> use or interest.
>
> Nick
>
> On Fri, Jul 6, 2012 at 10:41 AM, Lucia R.Latino 
> <Latino@economia.uniroma2.it> wrote:
>> Dear Michal and Stephen,
>>
>> Thanks for your answer. I tried Michal's code and it actually 
>> recovers the log pseudolikelihood, however stata shows the value 
>> using again the scientific notation. Do you have any suggestion?
>> Furthermore, I am not sure I am able to write the same code for the 
>> other distributions I am using (lognfit, smfit, dagumfit, fisk, 
>> beta2). Do you think there is a way to simply change the format of 
>> the Iteration so to have more precision?
>>
>> If it can helps I copied the result I obtained.
>>
>> Thanks,
>> Lucia
>>
>> -----------------------------------------------------------start
>> output
>> ---------------------------------------------------------------------
>> -
>> ------
>> ------------
>> gb2fit dec_ae, cdf(gb2cdf) pdf(gb2pdf) svy
>>
>> initial:       log pseudolikelihood =     -<inf>  (could not be
evaluated)
>> feasible:      log pseudolikelihood = -1.469e+08
>> rescale:       log pseudolikelihood = -1.469e+08
>> rescale eq:    log pseudolikelihood = -1.075e+08
>> Iteration 0:   log pseudolikelihood = -1.075e+08  (not concave)
>> Iteration 1:   log pseudolikelihood = -1.064e+08  (not concave)
>> Iteration 2:   log pseudolikelihood = -1.063e+08
>> Iteration 3:   log pseudolikelihood = -1.060e+08  (not concave)
>> Iteration 4:   log pseudolikelihood = -1.060e+08
>> Iteration 5:   log pseudolikelihood = -1.059e+08
>> Iteration 6:   log pseudolikelihood = -1.059e+08
>> Iteration 7:   log pseudolikelihood = -1.059e+08
>> Iteration 8:   log pseudolikelihood = -1.059e+08
>> Iteration 9:   log pseudolikelihood = -1.059e+08
>> Iteration 10:  log pseudolikelihood = -1.059e+08 Iteration 11:  log 
>> pseudolikelihood = -1.059e+08 Iteration 12:  log pseudolikelihood =
>> -1.059e+08
>>
>> ML fit of GB2 distribution
>>
>> pweight:  iwght                                   Number of obs    =
>> 11183
>> Strata:   <one>                                   Number of strata =
>> 30
>> PSU:      <observations>                          Number of PSUs   =
>> 564
>>                                                   Population size  =
>> 12272397
>>                                                   F(   0,    535)  =
>> .
>>                                                   Prob > F         =
>> .
>>
>> ---------------------------------------------------------------------
>> -
>> ------
>> --
>>       dec_ae |      Coef.   Std. Err.      t    P>|t|     [95% Conf.
>> Interval]
>> -------------+-------------------------------------------------------
>> -------------+-
>> -------------+------
>> --
>> a            |
>>        _cons |   2.327935   .2782045     8.37   0.000     1.781426
>> 2.874444
>> -------------+-------------------------------------------------------
>> -------------+-
>> -------------+------
>> --
>> b            |
>>        _cons |   3997.361   250.2252    15.98   0.000     3505.814
>> 4488.907
>> -------------+-------------------------------------------------------
>> -------------+-
>> -------------+------
>> --
>> p            |
>>        _cons |   1.603659   .2993526     5.36   0.000     1.015606
>> 2.191712
>> -------------+-------------------------------------------------------
>> -------------+-
>> -------------+------
>> --
>> q            |
>>        _cons |   2.983855   .6731986     4.43   0.000     1.661412
>> 4.306297
>> ---------------------------------------------------------------------
>> -
>> ------
>> --
>>
>>  tempvar ll
>> local wv= "`e(wvar)'"
>>
>>  gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)*
>> log(dec_ae) ///
>>          - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - ///
>>          (e(bp)+e(bq))*log(1+(dec_ae/e(bb))^e(ba))
>>
>>  qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum)
>>
>> -1.059e+08
>> -----------------------------------------------------------end output
>> ---------------------------------------------------------------------
>> -
>> ------
>> ------------
>>
>> Lucia
>>
>>
>> -----Messaggio originale-----
>> Da: owner-statalist@hsphsun2.harvard.edu
>> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Michal 
>> Brzezinski
>> Inviato: giovedì 5 luglio 2012 23:15
>> A: statalist@hsphsun2.harvard.edu
>> Oggetto: Re: st: change the format of the pseudo log-likelihood in 
>> gb2fit
>>
>> You can recover pseudo log-likelihood writing an explicit expression 
>> for log-likelihood for a given model.
>> For example, in case of GB2 model you could try the following code:
>>
>> gb2fit dec, cdf(gb2cdf) pdf(gb2pdf) svy tempvar ll local wv= 
>> "`e(wvar)'"
>>
>> gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)* 
>> log(dec)
> ///
>>         - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - ///
>>         (e(bp)+e(bq))*log(1+(dec/e(bb))^e(ba))
>> qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum)
>>
>> ----------
>> Hope this helps,
>> Michal
>>
>> 2012/7/5 Lucia Latino <Latino@economia.uniroma2.it>:
>>> Dear Statalist,
>>>
>>> After running - gb2fit - smfit - lognfit - Stata reports the pseudo 
>>> log-likelihood with the scientific notation, but I want to view the 
>>> full likelihood.
>>> Usually, I can obtain the full log-likelihood by using - display
>>> e(ll)
>>> -
>>>
>>> However, I need to add the option svy for the estimation (e.g. - 
>>> gb2fit dec, stats cdf(gb2cdf) pdf(gb2pdf) svy - ) and after adding 
>>> the option 'svy', Stata doesn't store anymore the pseudo 
>>> log-likelihood in
>> e(ll).
>>>
>>> Is there any way to do change the format of the pseudo 
>>> log-likelihood in the iterations or a way to obtain it after the
estimation?

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/