Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Generalized lineal models with survey data
From
Paolina Medina <[email protected]>
To
[email protected]
Subject
Re: st: Generalized lineal models with survey data
Date
Tue, 27 Jul 2010 12:32:01 -0500
Thank you very much for your useful and enlightening answer!!
If you let me, i would like to ask you just 2 more questions:
Why do you think this very large quasi log likelihood value is
arising? Could it be just because i am using a lot of dummy variables?
Or may be just my regressors are not the best predictors for the
dependent variable?
And, finally if you were to choose between the nbreg regression that i
just posted and the following poisson regression with the same
regressors, which one would you choose and why? Or which test would
you run to answer that question? The thing is that i think there is no
gof test available for this regressions with survey data in stata. Or
is it?
Again, thank you very very much for your patience and everything.
Poisson regression:
svy: poisson ncels resmay6 numradios nTVs tfijo tpaga luz ncompus
internet prim2 sec2 prepa2 normal2 tec2 pro2 m2 doc2 tra
> bajadores e1 e2 e3 e4 e5 e6 e7 e8 e9 e10 e11 e12 e13 e14 e15 e16 e17 e18 e19 e20 e21 e22 e23 e24 e25 e26 e27 e28 e29 e30 e3
> 1 e32 estrato1 estrato2 estrato3 estrato4, log
(running poisson on estimation sample)
note: e3 dropped due to collinearity
note: estrato2 dropped due to collinearity
Iteration 0: log pseudolikelihood = -32297663
Iteration 1: log pseudolikelihood = -32264546
Iteration 2: log pseudolikelihood = -32264459
Iteration 3: log pseudolikelihood = -32264459
Poisson regression Number of obs = 6089
LR chi2(51) = 1.761e+07
Prob > chi2 = 0.0000
Log pseudolikelihood = -32264459 Pseudo R2 = 0.2143
------------------------------------------------------------------------------
ncels | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
resmay6 | -.1189499 .0003869 -307.48 0.000 -.1197081 -.1181917
numradios | .0516967 .0002443 211.65 0.000 .051218 .0521754
nTVs | .1765135 .0001923 918.12 0.000 .1761367 .1768903
tfijo | -.1133193 .0004484 -252.70 0.000 -.1141982 -.1124404
tpaga | .1687625 .0004486 376.22 0.000 .1678833 .1696417
luz | .760426 .0046124 164.87 0.000 .7513859 .7694661
ncompus | .1401203 .0003325 421.45 0.000 .1394686 .1407719
internet | -.0047513 .0005739 -8.28 0.000 -.0058761 -.0036265
prim2 | .1220794 .0004182 291.90 0.000 .1212596 .1228991
sec2 | .1243139 .0002344 530.28 0.000 .1238544 .1247733
prepa2 | .0985697 .0002857 344.95 0.000 .0980097 .0991298
normal2 | -.0754923 .0007591 -99.45 0.000 -.0769801 -.0740045
tec2 | .0036673 .0007941 4.62 0.000 .002111 .0052237
pro2 | .0751998 .0004025 186.83 0.000 .0744109 .0759886
m2 | .0292362 .0007304 40.03 0.000 .0278047 .0306678
doc2 | -.1606452 .002035 -78.94 0.000 -.1646338 -.1566567
trabajadores | .1317087 .0001916 687.47 0.000 .1313332 .1320842
e1 | .0016868 .002384 0.71 0.479 -.0029858 .0063595
e2 | .0648902 .002022 32.09 0.000 .0609271 .0688533
e4 | -.0181873 .0026577 -6.84 0.000 -.0233962 -.0129784
e5 | -.1868012 .0021267 -87.83 0.000 -.1909695 -.1826329
e6 | .1466339 .0026388 55.57 0.000 .141462 .1518058
e7 | -.3935259 .0021961 -179.20 0.000 -.3978301 -.3892217
e8 | -.1245186 .0020253 -61.48 0.000 -.1284882 -.120549
e9 | -.2299049 .0019112 -120.29 0.000 -.2336509 -.226159
e10 | -.083798 .0022478 -37.28 0.000 -.0882035 -.0793924
e11 | -.3531031 .0020698 -170.60 0.000 -.3571598 -.3490464
e12 | -.4600155 .00238 -193.28 0.000 -.4646802 -.4553507
e13 | -.3038536 .002392 -127.03 0.000 -.3085418 -.2991653
e14 | -.1274274 .0019299 -66.03 0.000 -.13121 -.1236447
e15 | -.3696341 .0019058 -193.95 0.000 -.3733694 -.3658987
e16 | -.0019455 .0020393 -0.95 0.340 -.0059423 .0020514
e17 | -.1287657 .0023403 -55.02 0.000 -.1333526 -.1241788
e18 | -.106628 .0026009 -41.00 0.000 -.1117257 -.1015304
e19 | -.1713332 .0019934 -85.95 0.000 -.1752401 -.1674263
e20 | -.3297292 .0023891 -138.02 0.000 -.3344117 -.3250467
e21 | -.2838863 .0020318 -139.72 0.000 -.2878685 -.2799042
e22 | -.079044 .0022371 -35.33 0.000 -.0834287 -.0746593
e23 | .1517949 .0022013 68.96 0.000 .1474803 .1561094
e24 | -.2631791 .0022394 -117.52 0.000 -.2675682 -.25879
e25 | .0587084 .0021346 27.50 0.000 .0545247 .0628921
e26 | .0442957 .0020797 21.30 0.000 .0402195 .0483719
e27 | .1651334 .0021816 75.69 0.000 .1608575 .1694092
e28 | -.0281828 .0020233 -13.93 0.000 -.0321484 -.0242172
e29 | -.6082313 .002915 -208.66 0.000 -.6139446 -.6025181
e30 | -.144252 .0020102 -71.76 0.000 -.1481918 -.1403121
e31 | .0250983 .0022023 11.40 0.000 .0207819 .0294146
e32 | -.1352466 .0025166 -53.74 0.000 -.1401789 -.1303142
estrato1 | .189495 .0005579 339.69 0.000 .1884016 .1905883
estrato3 | -.322916 .0010167 -317.62 0.000 -.3249087 -.3209234
estrato4 | -.4792419 .0008096 -591.98 0.000 -.4808286 -.4776552
_cons | -1.493952 .0049766 -300.19 0.000 -1.503706 -1.484198
------------------------------------------------------------------------------
Computing scores...
Survey results:
Survey: Poisson regression
Number of strata = 4 Number of obs = 6089
Number of PSUs = 837 Population size = 27782772
Design df = 833
F( 51, 783) = 50.61
Prob > F = 0.0000
------------------------------------------------------------------------------
| Linearized
ncels | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
resmay6 | -.1189499 .0302713 -3.93 0.000 -.1783669 -.0595329
numradios | .0516967 .019889 2.60 0.010 .0126581 .0907352
nTVs | .1765135 .0210548 8.38 0.000 .1351867 .2178402
tfijo | -.1133193 .0389423 -2.91 0.004 -.1897558 -.0368828
tpaga | .1687625 .0330961 5.10 0.000 .1038009 .233724
luz | .760426 .4341483 1.75 0.080 -.0917272 1.612579
ncompus | .1401203 .0232643 6.02 0.000 .0944568 .1857837
internet | -.0047513 .0434178 -0.11 0.913 -.0899725 .0804699
prim2 | .1220794 .0353386 3.45 0.001 .0527162 .1914425
sec2 | .1243139 .0272578 4.56 0.000 .0708118 .177816
prepa2 | .0985697 .0253691 3.89 0.000 .0487748 .1483647
normal2 | -.0754923 .0678928 -1.11 0.266 -.2087533 .0577688
tec2 | .0036673 .0752994 0.05 0.961 -.1441316 .1514663
pro2 | .0751998 .038429 1.96 0.051 -.0002293 .1506288
m2 | .0292362 .04043 0.72 0.470 -.0501204 .1085929
doc2 | -.1606452 .0938375 -1.71 0.087 -.344831 .0235405
trabajadores | .1317087 .0193343 6.81 0.000 .093759 .1696583
e1 | .0016868 .1667689 0.01 0.992 -.3256499 .3290235
e2 | .0648902 .147677 0.44 0.660 -.2249727 .3547531
e4 | -.0181873 .1666332 -0.11 0.913 -.3452575 .3088829
e5 | -.1868012 .1583675 -1.18 0.239 -.4976475 .1240451
e6 | .1466339 .1557807 0.94 0.347 -.159135 .4524028
e7 | -.3935259 .2460116 -1.60 0.110 -.8764014 .0893497
e8 | -.1245186 .1549826 -0.80 0.422 -.4287209 .1796838
e9 | -.2299049 .1487182 -1.55 0.123 -.5218113 .0620015
e10 | -.083798 .2982254 -0.28 0.779 -.6691596 .5015637
e11 | -.3531031 .1536798 -2.30 0.022 -.6547482 -.0514579
e12 | -.4600155 .1721164 -2.67 0.008 -.7978484 -.1221826
e13 | -.3038536 .1661253 -1.83 0.068 -.629927 .0222199
e14 | -.1274274 .1614396 -0.79 0.430 -.4443035 .1894488
e15 | -.3696341 .151954 -2.43 0.015 -.6678918 -.0713763
e16 | -.0019455 .2189627 -0.01 0.993 -.4317289 .4278379
e17 | -.1287657 .1711779 -0.75 0.452 -.4647564 .207225
e18 | -.106628 .162161 -0.66 0.511 -.4249202 .2116641
e19 | -.1713332 .1540608 -1.11 0.266 -.4737261 .1310597
e20 | -.3297292 .247216 -1.33 0.183 -.8149688 .1555104
e21 | -.2838863 .1542873 -1.84 0.066 -.5867239 .0189512
e22 | -.079044 .1822687 -0.43 0.665 -.436804 .278716
e23 | .1517949 .1548563 0.98 0.327 -.1521596 .4557493
e24 | -.2631791 .179553 -1.47 0.143 -.6156086 .0892503
e25 | .0587084 .176449 0.33 0.739 -.2876286 .4050453
e26 | .0442957 .1611852 0.27 0.784 -.2720812 .3606725
e27 | .1651334 .168757 0.98 0.328 -.1661055 .4963723
e28 | -.0281828 .1619115 -0.17 0.862 -.3459853 .2896197
e29 | -.6082313 .1689908 -3.60 0.000 -.9399292 -.2765335
e30 | -.144252 .190234 -0.76 0.448 -.5176462 .2291423
e31 | .0250983 .1760391 0.14 0.887 -.3204341 .3706306
e32 | -.1352466 .2260741 -0.60 0.550 -.5789884 .3084952
estrato1 | .189495 .0642525 2.95 0.003 .0633791 .3156108
estrato3 | -.322916 .111296 -2.90 0.004 -.5413697 -.1044624
estrato4 | -.4792419 .1214264 -3.95 0.000 -.7175797 -.2409042
_cons | -1.493952 .462237 -3.23 0.001 -2.401238 -.5866655
------------------------------------------------------------------------------
On Tue, Jul 27, 2010 at 12:02 PM, Stas Kolenikov <[email protected]> wrote:
> No, you don't have any problems with the degrees of freedom, which is
> #PSUs - #strata = 837-4 = 833, and is reported as such. So I tend to
> believe in Steven's story about empirical underidentification of the
> overdispersion parameter: the likelihood is so flat in alpha that the
> curvature (inverse of the variance) of the likelihood wrt this
> parameter cannot be estimated with numeric accuracy that Stata would
> find acceptable to report. And yes, this is an indication that
> overdispersion is not such a great problem: coniditioning on
> covariates and taking weights into account seems to make your data
> approximately OK.
>
> As for the general convergence problems, they may be caused by the
> scale of weights. Note that your log pseudo-likelihood has 8 digits
> before the decimal point, and typically Stata wants to optimize things
> down to 7 or so digits after the decimal point, that is, you need to
> have about 15 reliable digits to declare convergence. That's too much
> to ask for, as 15 digits is the accuracy limit of the -datatype-
> double. In this situation (and in this situation only), it would be OK
> to relax the convergence criteria by specifying something like
> -ltolerance(1e-3)- instead of the default 1e-7; or rescale the weights
> so that they sum up to say sample size rather than the population
> size.
>
> On Tue, Jul 27, 2010 at 5:30 PM, Paolina Medina
> <[email protected]> wrote:
>> Thank you both, very much.
>> So this almost zero alpha, without a confidence interval can be taken
>> to indicate that there is no overdispersion in the model?
>> Here is my svyset statement and the complete output..
>> I am using 52 regressors (including the constant), i really dont know
>> how many are the design degrees of freedom... But in fact whenever i
>> take any of these regressors i get a lot of troubles with convergence
>> in the survey results (not concave or backed up) and i have to throw
>> away many other regressors to get convergence again.
>> Do you know anything i can do to fix this?
>
> --
> Stas Kolenikov, also found at http://stas.kolenikov.name
> Small print: I use this email account for mailing lists only.
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
--
Paolina Medina Palma
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/