Re: st: too good to be true : lr test in mlogit?

[John Litfiba <cariboupad@gmx.fr>]

Thank you so much for your kind and precise answer, Nick
I would be definitively interested if by chance you have in mind a
paper that discuss the large sample side effects on P-values that you
mention

Best Regards,

On 13 May 2011 10:31, Nick Cox <njcoxstata@gmail.com> wrote:
> I see nothing surprising here. The likelihood is the product of many
> very small probabilities, so will be very small overall. The P-values
> are at least in part a side-effect of using a very large sample size.
> I don't know that a model with such a low pseudo-R-square is "too good
> to be true", but it depends on your expectations. If this is analysis
> of data on people, as I wildly guess, the Maarten Buis argument that
> high levels of "explanation" are not to be expected given what else we
> know about the many determinants and influences on human behaviour
> could apply.
>
> Present-day significance testing machinery was largely designed in the
> first few decades of the 20th century to safeguard natural scientists
> against over-interpreting results from very small samples. Present-day
> social scientists in the early 21st century need other measures to
> safeguard themselves against over-interpreting significance tests from
> very large samples.
>
> On Fri, May 13, 2011 at 9:12 AM, John Litfiba <cariboupad@gmx.fr> wrote:
>> Dear all (again)
>>
>> I was wondering if my results seems too good to be true. I run a
>> multinomial logit for yvar (caterical variable with 4 possible values)
>> and I obtain the following results :
>>
>> 1) It is normal to obtain such a negative log likelihood when we use
>> very large sample, right ? (n=2 millions here)
>> 2) if the association (for example given by tabulation) show that
>> there is strong association between yvar and xvar1 then it is
>> plausible to obtain this fastastic LR statistic of... 140000 ??
>>
>>
>> Many many thanks in advance
>>
>> mlogit yvar xvar1 xvar2
>>
>> Iteration 0:   log likelihood = -1953742.5
>> Iteration 1:   log likelihood =   -1900152
>> Iteration 2:   log likelihood = -1883338.4
>> Iteration 3:   log likelihood =   -1880317
>> Iteration 4:   log likelihood = -1880312.7
>> Iteration 5:   log likelihood = -1880312.7
>>
>> Multinomial logistic regression                   Number of obs   =    2227058
>>                                                  LR chi2(6)      =  146859.43
>>                                                  Prob > chi2     =     0.0000
>> Log likelihood = -1880312.7                       Pseudo R2       =     0.0376
>>
>> ------------------------------------------------------------------------------
>>       order |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
>> -------------+----------------------------------------------------------------
>> yvar0 |  (base outcome)
>> -------------+----------------------------------------------------------------
>> yvar1           |
>>      xvar1 |  -2.137044   .0104876  -203.77   0.000    -2.157599   -2.116489
>>      xvar2|  -.0099444   .0001223   -81.32   0.000    -.0101841   -.0097047
>>       _cons |   1.708873   .0125759   135.88   0.000     1.684225    1.733522
>> -------------+----------------------------------------------------------------
>> yvar2         |
>>      xvar1 |   .8905294   .0734511    12.12   0.000     .7465678    1.034491
>>      xvar2 |  -.0087927   .0003393   -25.92   0.000    -.0094576   -.0081277
>>       _cons |  -3.672227   .0758592   -48.41   0.000    -3.820908   -3.523546
>> -------------+----------------------------------------------------------------
>> yvar3          |
>>      xvar1 |  -3.826486   .0113315  -337.69   0.000    -3.848695   -3.804276
>>      xvar2 |  -.0054125   .0002488   -21.76   0.000    -.0059002   -.0049249
>>       _cons |   1.244583   .0180673    68.89   0.000     1.209171    1.279994
>> ------------------------------------------------------------------------------
>> *
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/