Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | "Lucia R.Latino" <Latino@economia.uniroma2.it> |
| To | <statalist@hsphsun2.harvard.edu> |
| Subject | R: st: change the format of the pseudo log-likelihood in gb2fit |
| Date | Sat, 7 Jul 2012 16:11:41 +0200 |
Thanks Nick! Best, Lucia -----Messaggio originale----- Da: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Nick Cox Inviato: venerdì 6 luglio 2012 23:38 A: statalist@hsphsun2.harvard.edu Oggetto: Re: st: change the format of the pseudo log-likelihood in gb2fit Thanks for clarifying what you want. On your questions others may have answers; here are mine. 1. My guess is No. This concerns not programs like -lognfit- but -ml- that does the work. This is just a guess. 2. Michal has in effect already addressed this. You need to write your own code to get what you want. 3. No comment. Nick On Fri, Jul 6, 2012 at 9:47 PM, Lucia R.Latino <Latino@economia.uniroma2.it> wrote: > Dear Nick, > > I am sorry if I was not clear in my previous email. I will try to > explain here better. > > I need to choose the best fit for my data on consumption and I am > using survey data. > > I have been using the programs written by Stephen Jenkins (e.g. > gb2fit, smfit, dagumfit, lognfit) using the option svy. > > Stata shows the log pseudolikelihood with the scientific notation and > it does not store the loglikelihood in e(ll). > > I used the code kindly suggested from Michal Brzezinski to recover the > log pseudolikelihood for the gb2fit and I tried your suggestion of > using > -display- with a different format on r(sum). It worked. > > In order to get the full likelihood also for the other distributions (e.g. > lognormal, Singh-Maddala, Dagum, Fisk, beta2) following the procedure > above, I would need, as you said, to write a code for the log of each > distribution functions. It's a little bit complicated (for me, at least). > > So, I was wondering if: > 1. there is a way to change the format of the output of -lognfit-, > -smfit-, -dagumfit-, -fisk-, -beta2- in such way that the iterations > of the log pseudolikelihood are shown not in scientific notation; or > 2. there is an easier way to recover the log pseudolikelihood and > store it after I run each command to fit the distribution; or 3. there > is someone who can help me writing the code for the log of each > distribution functions (which I understand it may be just asking too > much), similarly to what Michal Brzezinski did for the gb2. > > Best, > Lucia > > > > > > -----Messaggio originale----- > Da: owner-statalist@hsphsun2.harvard.edu > [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Nick Cox > Inviato: venerdì 6 luglio 2012 20:19 > A: statalist@hsphsun2.harvard.edu > Oggetto: Re: st: change the format of the pseudo log-likelihood in > gb2fit > > The last bit involves taking logarithms of some slightly messy expressions. > That's all. > > On your first point, I don't understand what you are missing. If the > final log-likelihoods for different models are the same to 4 > significant figures, I suggest that they are to be regarded as identical in practice. > Alternatively, I guess that least one of the following statements is true. > > 1. You didn't try my suggestion of using -display- with a different > format on r(sum). > > 2. You did try it, but ran into some problem, in which case you need > to spell out what that problem is. > > 3. You want something else, but again you need to spell out what that is. > That said, it seems very unlikely that anyone will rewrite these > programs for you. > > Nick > > On Fri, Jul 6, 2012 at 6:59 PM, Lucia R.Latino > <Latino@economia.uniroma2.it> > wrote: >> Dear Nick, >> >> thanks for your hint. >> >> I am trying to compare different nested distributions and I would >> love to compare the log likelihood. With the scientific notation I >> cannot see any difference. That is the reason why I was trying to >> obtain the full log likelihood. >> >> Combine Michael's code and -display- I am able to get the full log >> likelihood. But I haven't been able to change how stata display the >> result after gb2fit or similar command. >> >> Does anyone can help me in writing the explicit expression for >> log-likelihood for lognfit, dagumfit, smfit, fisk and Beta2 similar >> to what Michal wrote for the gb2fit? >> >> Thanks, >> Lucia >> >> -----Messaggio originale----- >> Da: owner-statalist@hsphsun2.harvard.edu >> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Nick Cox >> Inviato: venerdì 6 luglio 2012 11:58 >> A: statalist@hsphsun2.harvard.edu >> Oggetto: Re: st: change the format of the pseudo log-likelihood in >> gb2fit >> >> -gb2fit- is a user-written program from SSC written by Stephen Jenkins. >> >> -display- takes a format which can be used to change how a result is >> displayed. Look at the help for -display- and for -format-. However, >> it is difficult to see that further decimal places could be of either >> use or interest. >> >> Nick >> >> On Fri, Jul 6, 2012 at 10:41 AM, Lucia R.Latino >> <Latino@economia.uniroma2.it> wrote: >>> Dear Michal and Stephen, >>> >>> Thanks for your answer. I tried Michal's code and it actually >>> recovers the log pseudolikelihood, however stata shows the value >>> using again the scientific notation. Do you have any suggestion? >>> Furthermore, I am not sure I am able to write the same code for the >>> other distributions I am using (lognfit, smfit, dagumfit, fisk, >>> beta2). Do you think there is a way to simply change the format of >>> the Iteration so to have more precision? >>> >>> If it can helps I copied the result I obtained. >>> >>> Thanks, >>> Lucia >>> >>> -----------------------------------------------------------start >>> output >>> -------------------------------------------------------------------- >>> - >>> - >>> ------ >>> ------------ >>> gb2fit dec_ae, cdf(gb2cdf) pdf(gb2pdf) svy >>> >>> initial: log pseudolikelihood = -<inf> (could not be > evaluated) >>> feasible: log pseudolikelihood = -1.469e+08 >>> rescale: log pseudolikelihood = -1.469e+08 >>> rescale eq: log pseudolikelihood = -1.075e+08 >>> Iteration 0: log pseudolikelihood = -1.075e+08 (not concave) >>> Iteration 1: log pseudolikelihood = -1.064e+08 (not concave) >>> Iteration 2: log pseudolikelihood = -1.063e+08 >>> Iteration 3: log pseudolikelihood = -1.060e+08 (not concave) >>> Iteration 4: log pseudolikelihood = -1.060e+08 >>> Iteration 5: log pseudolikelihood = -1.059e+08 >>> Iteration 6: log pseudolikelihood = -1.059e+08 >>> Iteration 7: log pseudolikelihood = -1.059e+08 >>> Iteration 8: log pseudolikelihood = -1.059e+08 >>> Iteration 9: log pseudolikelihood = -1.059e+08 >>> Iteration 10: log pseudolikelihood = -1.059e+08 Iteration 11: log >>> pseudolikelihood = -1.059e+08 Iteration 12: log pseudolikelihood = >>> -1.059e+08 >>> >>> ML fit of GB2 distribution >>> >>> pweight: iwght Number of obs = >>> 11183 >>> Strata: <one> Number of strata = >>> 30 >>> PSU: <observations> Number of PSUs = >>> 564 >>> Population size = >>> 12272397 >>> F( 0, 535) = >>> . >>> Prob > F = >>> . >>> >>> -------------------------------------------------------------------- >>> - >>> - >>> ------ >>> -- >>> dec_ae | Coef. Std. Err. t P>|t| [95% Conf. >>> Interval] >>> -------------+------------------------------------------------------ >>> -------------+- >>> -------------+- >>> -------------+------ >>> -- >>> a | >>> _cons | 2.327935 .2782045 8.37 0.000 1.781426 >>> 2.874444 >>> -------------+------------------------------------------------------ >>> -------------+- >>> -------------+- >>> -------------+------ >>> -- >>> b | >>> _cons | 3997.361 250.2252 15.98 0.000 3505.814 >>> 4488.907 >>> -------------+------------------------------------------------------ >>> -------------+- >>> -------------+- >>> -------------+------ >>> -- >>> p | >>> _cons | 1.603659 .2993526 5.36 0.000 1.015606 >>> 2.191712 >>> -------------+------------------------------------------------------ >>> -------------+- >>> -------------+- >>> -------------+------ >>> -- >>> q | >>> _cons | 2.983855 .6731986 4.43 0.000 1.661412 >>> 4.306297 >>> -------------------------------------------------------------------- >>> - >>> - >>> ------ >>> -- >>> >>> tempvar ll >>> local wv= "`e(wvar)'" >>> >>> gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)* >>> log(dec_ae) /// >>> - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - /// >>> (e(bp)+e(bq))*log(1+(dec_ae/e(bb))^e(ba)) >>> >>> qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum) >>> >>> -1.059e+08 >>> -----------------------------------------------------------end >>> output >>> -------------------------------------------------------------------- >>> - >>> - >>> ------ >>> ------------ >>> >>> Lucia >>> >>> >>> -----Messaggio originale----- >>> Da: owner-statalist@hsphsun2.harvard.edu >>> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Michal >>> Brzezinski >>> Inviato: giovedì 5 luglio 2012 23:15 >>> A: statalist@hsphsun2.harvard.edu >>> Oggetto: Re: st: change the format of the pseudo log-likelihood in >>> gb2fit >>> >>> You can recover pseudo log-likelihood writing an explicit expression >>> for log-likelihood for a given model. >>> For example, in case of GB2 model you could try the following code: >>> >>> gb2fit dec, cdf(gb2cdf) pdf(gb2pdf) svy tempvar ll local wv= >>> "`e(wvar)'" >>> >>> gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)* >>> log(dec) >> /// >>> - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - /// >>> (e(bp)+e(bq))*log(1+(dec/e(bb))^e(ba)) >>> qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum) >>> >>> ---------- >>> Hope this helps, >>> Michal >>> >>> 2012/7/5 Lucia Latino <Latino@economia.uniroma2.it>: >>>> Dear Statalist, >>>> >>>> After running - gb2fit - smfit - lognfit - Stata reports the pseudo >>>> log-likelihood with the scientific notation, but I want to view the >>>> full likelihood. >>>> Usually, I can obtain the full log-likelihood by using - display >>>> e(ll) >>>> - >>>> >>>> However, I need to add the option svy for the estimation (e.g. - >>>> gb2fit dec, stats cdf(gb2cdf) pdf(gb2pdf) svy - ) and after adding >>>> the option 'svy', Stata doesn't store anymore the pseudo >>>> log-likelihood in >>> e(ll). >>>> >>>> Is there any way to do change the format of the pseudo >>>> log-likelihood in the iterations or a way to obtain it after the > estimation? * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/