Re: st: mlogtest after mlogit

[Nick Cox <njcoxstata@gmail.com>]

Now we are really getting somewhere. "__000000" is a temporary name;
in essence it surely does not belong there. I guess now that you have
tickled a bug in one of the -spost- routines and, as recommended
already in this thread, you should bring this to the attention of the
authors, who as said are not active members of Statalist.

Note that -_pecats- does find all your categories 1/9 which supports
my earlier wild conjecture that their frequencies were not an issue.

So, the warning message itself now appears to be an error, but there
is no obvious reason to credit the rest of the output if -mlogtest- is
misunderstanding what is going on with your model and data. You did
not comment on the fact that -r(refval) is missing in your output, so
other difficulties may lie undetected.

Nick

On Tue, Oct 25, 2011 at 12:18 PM, Chiara Mussida <cmussida@gmail.com> wrote:
> On 25 October 2011 12:57, Nick Cox <njcoxstata@gmail.com> wrote:
>> If you look at the code of -mlogtest- the warning message that worries
>> you is when a helper program -_pecats- produces output that is
>> inconsistent. After your -mlogit- call you can run -_pecats- followed
>> by -return list-. In a well-behaved case, you will get output like
>> this in which r(numcats) matches the number of elements in r(catvals).
>> Your output will probably not match up. I don't know why that might
>> be, but I doubt that the category frequencies are somehow the issue.
>> (For "STATA" read "Stata".)
>>
>> . _pecats
>>
>> . return list
>>
>> scalars:
>>            r(numcats) =  9
>>             r(refval) =  3
>>
>> macros:
>>            r(catnms8) : "1 2 4 5 6 7 8 9 3"
>>            r(catvals) : "1 2 4 5 6 7 8 9 3"
>>             r(catnms) : "1 2 4 5 6 7 8 9 3"
>>              r(refnm) : "3"
>>
> Exactly Nick and thanks: when I type -_pecats- and -return list- I get:
>
>  return list
>
> scalars:
>            r(numcats) =  9
>             r(refval) =  .
>
> macros:
>            r(catnms8) : "1 2 3 4 5 6 7 8 9"
>            r(catvals) : "1 2 3 4 5 6 7 8 9 __000000"
>             r(catnms) : "1 2 3 4 5 6 7 8 9"
>
> where as you correctly assumed, I do not have the match between
> r(numcats) and  r(catvals), but something like __000000 which does not
> allow a perfect matching. I still do not know why it happens.
>
>
>
>
>
>>
>> On Tue, Oct 25, 2011 at 11:10 AM, Chiara Mussida <cmussida@gmail.com> wrote:
>>> On 25 October 2011 01:10, Richard Williams
>>> <richardwilliams.ndu@gmail.com> wrote:
>>>> At 05:59 AM 10/24/2011, Muhammad Anees wrote:
>>>>>
>>>>> have you tried -mlogtest, all- to verify if these categories have no
>>>>> other issues. Otherwise the test on -combine- might have resulted
>>>>> becaure some of the categories had small or no observations so
>>>>> checking the condition of equal coefficients from -comibe- did not
>>>>> work.
>>>>
>>>> I agree - I get nervous about using multiple-outcome commands like mlogit
>>>> with lots and lots of independent variables. You may be spreading the data
>>>> too thin. But, before taking this too much further, I'd like the original
>>>> poster to confirm that the most current version of mlogtest is indeed being
>>>> used. Otherwise we might be talking about a problem that was fixed 6 months
>>>> ago. Also, it might be good to present a frequency of the dependent
>>>> variable. Long and Freese's commands are sometimes pickier about coding than
>>>> Stata is, e.g. they sometimes don't like non-integer coding. Also, you would
>>>> see if some of the categories have very small frequency counts. Finally, I
>>>> would run a simple model with only one or two independent variables followed
>>>> by mlogtest. If the simple model works and the more complicated one doesn't,
>>>> that might indicate problems with one or more of the added variables or with
>>>> the data being spread too thin to do the test.
>>>>
>>>>
>>> Dear All, I confirm that my current version of mlogtest is and was the
>>> one indicated by Nick, precisely:
>>>
>>> . which mlogtest, all
>>>
>>> C:\Program Files\Stata12\ado\updates\m\mlogtest.ado
>>> *! version 1.7.6 jsl 2009-10-18
>>>
>>> in terms of model estimates, I guess that the issue is related to the
>>> relative frequency of my dependent variables categories':
>>>
>>>  ta transition
>>>
>>>  transition |      Freq.     Percent        Cum.
>>> ------------+-----------------------------------
>>>          1 |        271        0.70        0.70
>>>          2 |        132        0.34        1.04
>>>          3 |      1,119        2.90        3.94
>>>          4 |        379        0.98        4.93
>>>          5 |        722        1.87        6.80
>>>          6 |     13,959       36.17       42.97
>>>          7 |        388        1.01       43.98
>>>          8 |        168        0.44       44.41
>>>          9 |     21,450       55.59      100.00
>>> ------------+-----------------------------------
>>>      Total |     38,588      100.00
>>>
>>> e.g., categories 2 and 8 might be too small. Now, I cannot collapse my
>>> dep variable in a reduced number of categories and I hope that
>>> notwithstanding the STATA alert message after typing the test command
>>> (below I copied all the results) do not bias my results. ps: for the
>>> test N=25441 since the model estimates are referred to a subsample
>>> (aged 15-64) of the overall population (38588).
>>>
>>> mlogtest, c
>>>
>>> Problem determining number of categories.
>>>
>>> **** Wald tests for combining alternatives (N=25441)
>>>
>>>  Ho: All coefficients except intercepts associated with a given pair
>>>     of alternatives are 0 (i.e., alternatives can be combined).
>>>
>>> Alternatives tested|     chi2   df   P>chi2
>>> -------------------+------------------------
>>>       1-       2  |   18.576    5    0.002
>>>       1-       3  |    5.990    5    0.307
>>>       1-       4  |   13.565    5    0.019
>>>       1-       5  |  148.448 5 0.000
>>>       1-       6  |  178.434    5    0.000
>>>       1-       7  |   33.226    5    0.000
>>>       1-       8  |   68.938    5    0.000
>>>       1-       9  |  311.133    5    0.000
>>>       2-       3  |   19.911    5    0.001
>>>       2-       4  |   23.931    5    0.000
>>>       2-       5  |   68.619    5    0.000
>>>       2-       6  |   68.197    5    0.000
>>>       2-       7  |   24.027    5    0.000
>>>       2-       8  |   55.161    5    0.000
>>>       2-       9  |  127.081    5    0.000
>>>       3-       4  |   36.426    5    0.000
>>>       3-       5  |  438.391    5    0.000
>>>       3-       6  |  703.923    5    0.000
>>>       3-       7  |   59.553    5    0.000
>>>       3-       8  |  103.537    5    0.000
>>>       3-       9  | 1130.422    5    0.000
>>>       4-       5  |  197.002    5    0.000
>>>       4-       6  |  103.387    5    0.000
>>>       4-       7  |   91.631    5    0.000
>>>       4-       8  |  132.381    5    0.000
>>>       4-       9  |  530.943    5    0.000
>>>       5-       6  |  592.783    5    0.000
>>>       5-       7  |  192.566    5    0.000
>>>       5-       8  |  142.185    5    0.000
>>>       5-       9  |  281.162    5    0.000
>>>       6-       7  |  520.969    5    0.000
>>>       6-       8  |  430.911    5    0.000
>>>       6-       9  | 5946.722    5    0.000
>>>       7-       8  |   24.867    5    0.000
>>>       7-       9  |  231.357    5    0.000
>>>       8-       9  |   82.933    5    0.000
>>> --------------------------------------------
>>>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/