Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Problem with ZINB inflation equation estimates
From
James Shaw <[email protected]>
To
[email protected]
Subject
Re: st: Problem with ZINB inflation equation estimates
Date
Wed, 1 Sep 2010 16:04:48 -0500
Thanks for the prompt response. Yes, zero cells in the inflation
equation (corresponding to an absence of zero counts for groups
indicated by regressors with extreme negative estimates) would make
sense. However, as shown in Table 1 below, there are zero counts for
these groups. In the table, "0" indicates the number of zero count
observations, while "1" indicates the number of observations with a
count >=1.
This is a repeated-measures data set. Counts for the 24 states are
observed for the same set of individuals. The sandwich variance
estimator was used to account for person-level clustering. Without
application of the cluster-robust variance estimator, estimates of
standard errors for the extreme parameter estimates tended toward
infinity.
Additionally, I am wondering whether the extreme estimates are an
indication of the inability of the inflation model to capture
heterogeneity due to excess zeros after taking into account
individual-level heterogeneity, as reflected by alpha. The
zero-inflated Poisson (ZIP) model is comparatively well behaved (see
Table 2 below). After adjusting for individual-level heterogeneity,
the probability of some states having a zero count may approach zero.
Thus, ZINB may be unable to provide reliable estimates of the
corresponding inflation model parameters.
Based on your comments, it would appear that I should exclude ZINB
from consideration and focus on ZIP and negative binomial (without
zero inflation) instead.
--
Jim
TABLE 1
state 0 1
1 6,723 5,677
2 4,077 8,323
3 6,857 5,543
4 4,186 8,214
5 6,886 5,514
6 4,034 8,366
7 6,655 5,745
8 3,882 8,518
9 6,178 6,222
10 3,853 8,547
11 6,555 5,845
12 3,801 8,599
13 6,795 5,605
14 4,136 8,264
15 6,847 5,553
16 4,041 8,359
17 6,513 5,887
18 4,139 8,261
19 6,600 5,800
20 3,972 8,428
21 6,299 6,101
22 3,408 8,992
23 6,553 5,847
24 3,989 8,411
TABLE 2
| Robust
| IRR Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
nlic2
_Istate_2 .9865309 .0514006 -0.26 0.795 .8907607 1.092598
_Istate_3 1.051092 .0850525 0.62 0.538 .8969394 1.231739
_Istate_4 1.142369 .090406 1.68 0.093 .9782348 1.334043
_Istate_5 1.037862 .0817944 0.47 0.637 .8893154 1.21122
_Istate_6 1.111375 .0884138 1.33 0.184 .9509215 1.298903
_Istate_7 1.065085 .1035223 0.65 0.517 .8803406 1.2886
_Istate_8 1.039028 .0877759 0.45 0.650 .8804785 1.226128
_Istate_9 1.319903 .1289443 2.84 0.004 1.089899 1.598445
_Istate_10 1.239489 .111583 2.38 0.017 1.038998 1.478668
_Istate_11 1.236957 .1227598 2.14 0.032 1.018307 1.502556
_Istate_12 1.227542 .1110276 2.27 0.023 1.02813 1.465633
_Istate_13 .9693349 .0508785 -0.59 0.553 .8745728 1.074365
_Istate_14 .9100219 .0516526 -1.66 0.097 .8142128 1.017105
_Istate_15 1.164887 .0990005 1.80 0.073 .986149 1.376021
_Istate_16 1.106839 .093111 1.21 0.228 .9385959 1.30524
_Istate_17 1.129445 .0927545 1.48 0.138 .9615265 1.326689
_Istate_18 1.079823 .0878866 0.94 0.345 .9206048 1.266577
_Istate_19 1.00173 .0680997 0.03 0.980 .8767674 1.144504
_Istate_20 .9557701 .0558659 -0.77 0.439 .8523141 1.071784
_Istate_21 1.224653 .1024695 2.42 0.015 1.03942 1.442896
_Istate_22 1.271586 .1071609 2.85 0.004 1.077983 1.499959
_Istate_23 1.180238 .1004533 1.95 0.052 .9988986 1.394497
_Istate_24 1.154552 .0989566 1.68 0.094 .9760166 1.365746
nlop (exposure)
inflate
_Istate_2 -.6431842 .0884337 -7.27 0.000 -.8165111 -.4698574
_Istate_3 -.1121791 .159063 -0.71 0.481 -.4239368 .1995787
_Istate_4 -.6264557 .1642806 -3.81 0.000 -.9484399 -.3044716
_Istate_5 -.1062271 .1542732 -0.69 0.491 -.4085969 .1961428
_Istate_6 -.6813996 .1620086 -4.21 0.000 -.9989306 -.3638687
_Istate_7 -.0572668 .1481622 -0.39 0.699 -.3476594 .2331259
_Istate_8 -.714012 .1411174 -5.06 0.000 -.990597 -.4374269
_Istate_9 -.1271494 .163172 -0.78 0.436 -.4469608 .1926619
_Istate_10 -.6918837 .162056 -4.27 0.000 -1.009508 -.3742597
_Istate_11 -.0714629 .1640151 -0.44 0.663 -.3929267 .2500009
_Istate_12 -.7078715 .1596094 -4.44 0.000 -1.0207 -.3950428
_Istate_13 .0188447 .0861727 0.22 0.827 -.1500508 .1877401
_Istate_14 -.6176321 .0898303 -6.88 0.000 -.7936964 -.4415679
_Istate_15 .04241 .1593765 0.27 0.790 -.2699622 .3547823
_Istate_16 -.6410692 .162872 -3.94 0.000 -.9602924 -.321846
_Istate_17 -.1178183 .1591056 -0.74 0.459 -.4296597 .194023
_Istate_18 -.6074913 .1553598 -3.91 0.000 -.911991 -.3029916
_Istate_19 -.0944102 .1100006 -0.86 0.391 -.3100073 .121187
_Istate_20 -.6751463 .0927802 -7.28 0.000 -.8569921 -.4933004
_Istate_21 -.1707728 .1600858 -1.07 0.286 -.4845353 .1429897
_Istate_22 -.8657041 .1643409 -5.27 0.000 -1.187806 -.543602
_Istate_23 -.0754311 .163291 -0.46 0.644 -.3954756 .2446135
_Istate_24 -.6647537 .1589559 -4.18 0.000 -.9763015 -.3532059
_cons -.3606622 .1763068 -2.05 0.041 -.7062172 -.0151071
On Wed, Sep 1, 2010 at 3:25 PM, Stas Kolenikov <[email protected]> wrote:
> The -inflate- equation is essentially the logit model for the fixed
> zero. Excessive negative values probably mean that those states do not
> have any zeroes whatsoever. Stata tried to dig deep into this, never
> reaching minus infinity which would be the appropriate answer (recall
> "[this many] zeroes perfectly predicted" warning message from
> -logit-), but deciding that the value of -40 is a good enough numeric
> approximation to minus infinity. Of course when you try to do anything
> with this value, like computing the curvature of the likelihood
> function in the vicinity of this point, the exponent of -40 is
> indistinguishable from zero (specifically, two order of magnitudes
> smaller than c(epsdouble)), and hence the numeric error message.
>
> Of course I might have messed things up completely, in which case the
> direction of the effect is the opposite one: those states only have
> zeroes, so the model still needs to achieve perfect prediction by
> setting coefficients to infinity.
>
> On Wed, Sep 1, 2010 at 3:12 PM, James Shaw <[email protected]> wrote:
>> I am modeling count data that exhibit overdispersion and excess zeros
>> and have some concerns about the appropriateness of a zero-inflated
>> negative binomial (ZINB) model for the data. As can be seen from the
>> results below, a Wald test of alpha = 0 is significant, which suggests
>> that ZINB is more correctly specified than zero-inflated Poisson
>> (ZIP). However, the estimates for several of the regressors included
>> in the inflation equation take on extreme negative values, and tests
>> of the exponentiated estimates against zero cannot be performed. Is
>> this an indication of a problem with the ZINB model specification or
>> simply an issue with using Wald tests to evaluate nonlinear
>> transformations of the estimates? I am accustomed to seeing such
>> extreme estimates when modeling categorical outcomes with zero cells
>> or near-zero cells. However, I am not certain how to interpret
>> extreme estimates for the parameters of the inflation model.
>>
>> --
>> Jim
>>
>>
>> James W. Shaw, Ph.D., Pharm.D., M.P.H.
>> Assistant Professor
>> Department of Pharmacy Administration
>> College of Pharmacy
>> University of Illinois at Chicago
>> 833 South Wood Street, M/C 871, Room 252
>> Chicago, IL 60612
>> Tel.: 312-355-5666
>> Fax: 312-996-0868
>> Mobile Tel.: 215-852-3045
>>
>>
>>
>> | Robust
>> | IRR Std. Err. z P>|z| [95% Conf. Interval]
>> -------------+----------------------------------------------------------------
>> nlic2 |
>> _Istate_2 | 1.092649 .073559 1.32 0.188 .9575832 1.246767
>> _Istate_3 | 1.027985 .0974826 0.29 0.771 .853627 1.237956
>> _Istate_4 | 1.260222 .1113024 2.62 0.009 1.059911 1.498391
>> _Istate_5 | 1.015475 .0940208 0.17 0.868 .8469512 1.217532
>> _Istate_6 | 1.244825 .1100706 2.48 0.013 1.04675 1.480382
>> _Istate_7 | 1.059836 .1208843 0.51 0.610 .8475224 1.325337
>> _Istate_8 | 1.171791 .1071122 1.73 0.083 .9795862 1.401708
>> _Istate_9 | 1.345821 .1539067 2.60 0.009 1.075585 1.683953
>> _Istate_10 | 1.390382 .1343393 3.41 0.001 1.150511 1.680264
>> _Istate_11 | 1.237516 .144065 1.83 0.067 .9850502 1.554687
>> _Istate_12 | 1.384367 .1349304 3.34 0.001 1.143634 1.675774
>> _Istate_13 | .9682022 .0595369 -0.53 0.599 .8582697 1.092215
>> _Istate_14 | 1.002396 .0646459 0.04 0.970 .8833731 1.137456
>> _Istate_15 | 1.163044 .1160887 1.51 0.130 .9563875 1.414355
>> _Istate_16 | 1.229361 .119212 2.13 0.033 1.016572 1.486693
>> _Istate_17 | 1.127602 .1087036 1.25 0.213 .9334643 1.362115
>> _Istate_18 | 1.19585 .1122559 1.91 0.057 .994886 1.437408
>> _Istate_19 | .9935881 .0791236 -0.08 0.936 .8500053 1.161425
>> _Istate_20 | 1.067782 .0769608 0.91 0.363 .9271117 1.229797
>> _Istate_21 | 1.229869 .1205396 2.11 0.035 1.014922 1.49034
>> _Istate_22 | 1.488745 .1343797 4.41 0.000 1.247348 1.776859
>> _Istate_23 | 1.183237 .1180536 1.69 0.092 .9730741 1.43879
>> _Istate_24 | 1.287411 .1179763 2.76 0.006 1.075758 1.540707
>> nlop | (exposure)
>> -------------+----------------------------------------------------------------
>> inflate |
>> _Istate_2 | -15.18086 3.86403 -3.93 0.000 -22.75422 -7.607505
>> _Istate_3 | -.9113138 .8377267 -1.09 0.277 -2.553228 .7306003
>> _Istate_4 | -37.93109 3.212119 -11.81 0.000 -44.22673 -31.63545
>> _Istate_5 | -.8688422 .7869904 -1.10 0.270 -2.411315 .6736307
>> _Istate_6 | -34.5509 1.418593 -24.36 0.000 -37.33129 -31.77051
>> _Istate_7 | -.2903815 .7019179 -0.41 0.679 -1.666115 1.085352
>> _Istate_8 | -19.12953 1.197556 -15.97 0.000 -21.4767 -16.78236
>> _Istate_9 | -.3339708 .7646007 -0.44 0.662 -1.832561 1.164619
>> _Istate_10 | -16.91487 8.778676 -1.93 0.054 -34.12076 .2910193
>> _Istate_11 | -.2927704 .7666662 -0.38 0.703 -1.795408 1.209868
>> _Istate_12 | -40.69688 4.617249 -8.81 0.000 -49.74652 -31.64723
>> _Istate_13 | .0576077 .3917056 0.15 0.883 -.7101211 .8253366
>> _Istate_14 | -4.621435 20.1191 -0.23 0.818 -44.05415 34.81128
>> _Istate_15 | .1455226 .6589907 0.22 0.825 -1.146076 1.437121
>> _Istate_16 | -4.234189 12.39485 -0.34 0.733 -28.52765 20.05928
>> _Istate_17 | -.5810501 .7541463 -0.77 0.441 -2.05915 .8970495
>> _Istate_18 | -2.642428 2.516604 -1.05 0.294 -7.574882 2.290025
>> _Istate_19 | -.5181308 .4920146 -1.05 0.292 -1.482462 .4462001
>> _Istate_20 | -15.64851 2.008452 -7.79 0.000 -19.585 -11.71201
>> _Istate_21 | -.8170193 .8394646 -0.97 0.330 -2.46234 .8283011
>> _Istate_22 | -36.16489 .9060823 -39.91 0.000 -37.94078 -34.389
>> _Istate_23 | -.2927124 .7272732 -0.40 0.687 -1.718142 1.132717
>> _Istate_24 | -15.53286 7.341087 -2.12 0.034 -29.92112 -1.144592
>> _cons | -2.111847 .5767957 -3.66 0.000 -3.242346 -.9813481
>> -------------+----------------------------------------------------------------
>> /lnalpha | -.1629929 .1439461 -1.13 0.258 -.4451221 .1191363
>> -------------+----------------------------------------------------------------
>> alpha | .8495972 .1222962 .6407461 1.126523
>>
>> . testnl exp([lnalpha]_cons)=0
>>
>> (1) exp([lnalpha]_cons) = 0
>>
>> chi2(1) = 48.26
>> Prob > chi2 = 0.0000
>>
>> . testnl exp([inflate]_Istate_2)=1
>>
>> (1) exp([inflate]_Istate_2) = 1
>> warning: derivative with respect to inflate:_Istate_2
>> coefficient is near zero,
>> derivative treated as zero
>> Constraint (1) dropped
>>
>> chi2(0) = 0.00
>> Prob > chi2 = .
>>
>> . testnl exp([inflate]_Istate_4)=0
>>
>> (1) exp([inflate]_Istate_4) = 0
>> warning: derivative with respect to inflate:_Istate_4 coefficient is near zero,
>> derivative treated as zero
>> Constraint (1) dropped
>>
>> chi2(0) = 0.00
>> Prob > chi2 = .
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
>
>
>
> --
> Stas Kolenikov, also found at http://stas.kolenikov.name
> Small print: I use this email account for mailing lists only.
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
--
James W. Shaw, Ph.D., Pharm.D., M.P.H.
Assistant Professor
Department of Pharmacy Administration
College of Pharmacy
University of Illinois at Chicago
833 South Wood Street, M/C 871, Room 252
Chicago, IL 60612
Tel.: 312-355-5666
Fax: 312-996-0868
Mobile Tel.: 215-852-3045
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/