Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
From | Paolina Medina <carmencitamedina@gmail.com> |
To | statalist@hsphsun2.harvard.edu |
Subject | Re: st: Generalized lineal models with survey data |
Date | Tue, 27 Jul 2010 12:32:01 -0500 |
Thank you very much for your useful and enlightening answer!! If you let me, i would like to ask you just 2 more questions: Why do you think this very large quasi log likelihood value is arising? Could it be just because i am using a lot of dummy variables? Or may be just my regressors are not the best predictors for the dependent variable? And, finally if you were to choose between the nbreg regression that i just posted and the following poisson regression with the same regressors, which one would you choose and why? Or which test would you run to answer that question? The thing is that i think there is no gof test available for this regressions with survey data in stata. Or is it? Again, thank you very very much for your patience and everything. Poisson regression: svy: poisson ncels resmay6 numradios nTVs tfijo tpaga luz ncompus internet prim2 sec2 prepa2 normal2 tec2 pro2 m2 doc2 tra > bajadores e1 e2 e3 e4 e5 e6 e7 e8 e9 e10 e11 e12 e13 e14 e15 e16 e17 e18 e19 e20 e21 e22 e23 e24 e25 e26 e27 e28 e29 e30 e3 > 1 e32 estrato1 estrato2 estrato3 estrato4, log (running poisson on estimation sample) note: e3 dropped due to collinearity note: estrato2 dropped due to collinearity Iteration 0: log pseudolikelihood = -32297663 Iteration 1: log pseudolikelihood = -32264546 Iteration 2: log pseudolikelihood = -32264459 Iteration 3: log pseudolikelihood = -32264459 Poisson regression Number of obs = 6089 LR chi2(51) = 1.761e+07 Prob > chi2 = 0.0000 Log pseudolikelihood = -32264459 Pseudo R2 = 0.2143 ------------------------------------------------------------------------------ ncels | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- resmay6 | -.1189499 .0003869 -307.48 0.000 -.1197081 -.1181917 numradios | .0516967 .0002443 211.65 0.000 .051218 .0521754 nTVs | .1765135 .0001923 918.12 0.000 .1761367 .1768903 tfijo | -.1133193 .0004484 -252.70 0.000 -.1141982 -.1124404 tpaga | .1687625 .0004486 376.22 0.000 .1678833 .1696417 luz | .760426 .0046124 164.87 0.000 .7513859 .7694661 ncompus | .1401203 .0003325 421.45 0.000 .1394686 .1407719 internet | -.0047513 .0005739 -8.28 0.000 -.0058761 -.0036265 prim2 | .1220794 .0004182 291.90 0.000 .1212596 .1228991 sec2 | .1243139 .0002344 530.28 0.000 .1238544 .1247733 prepa2 | .0985697 .0002857 344.95 0.000 .0980097 .0991298 normal2 | -.0754923 .0007591 -99.45 0.000 -.0769801 -.0740045 tec2 | .0036673 .0007941 4.62 0.000 .002111 .0052237 pro2 | .0751998 .0004025 186.83 0.000 .0744109 .0759886 m2 | .0292362 .0007304 40.03 0.000 .0278047 .0306678 doc2 | -.1606452 .002035 -78.94 0.000 -.1646338 -.1566567 trabajadores | .1317087 .0001916 687.47 0.000 .1313332 .1320842 e1 | .0016868 .002384 0.71 0.479 -.0029858 .0063595 e2 | .0648902 .002022 32.09 0.000 .0609271 .0688533 e4 | -.0181873 .0026577 -6.84 0.000 -.0233962 -.0129784 e5 | -.1868012 .0021267 -87.83 0.000 -.1909695 -.1826329 e6 | .1466339 .0026388 55.57 0.000 .141462 .1518058 e7 | -.3935259 .0021961 -179.20 0.000 -.3978301 -.3892217 e8 | -.1245186 .0020253 -61.48 0.000 -.1284882 -.120549 e9 | -.2299049 .0019112 -120.29 0.000 -.2336509 -.226159 e10 | -.083798 .0022478 -37.28 0.000 -.0882035 -.0793924 e11 | -.3531031 .0020698 -170.60 0.000 -.3571598 -.3490464 e12 | -.4600155 .00238 -193.28 0.000 -.4646802 -.4553507 e13 | -.3038536 .002392 -127.03 0.000 -.3085418 -.2991653 e14 | -.1274274 .0019299 -66.03 0.000 -.13121 -.1236447 e15 | -.3696341 .0019058 -193.95 0.000 -.3733694 -.3658987 e16 | -.0019455 .0020393 -0.95 0.340 -.0059423 .0020514 e17 | -.1287657 .0023403 -55.02 0.000 -.1333526 -.1241788 e18 | -.106628 .0026009 -41.00 0.000 -.1117257 -.1015304 e19 | -.1713332 .0019934 -85.95 0.000 -.1752401 -.1674263 e20 | -.3297292 .0023891 -138.02 0.000 -.3344117 -.3250467 e21 | -.2838863 .0020318 -139.72 0.000 -.2878685 -.2799042 e22 | -.079044 .0022371 -35.33 0.000 -.0834287 -.0746593 e23 | .1517949 .0022013 68.96 0.000 .1474803 .1561094 e24 | -.2631791 .0022394 -117.52 0.000 -.2675682 -.25879 e25 | .0587084 .0021346 27.50 0.000 .0545247 .0628921 e26 | .0442957 .0020797 21.30 0.000 .0402195 .0483719 e27 | .1651334 .0021816 75.69 0.000 .1608575 .1694092 e28 | -.0281828 .0020233 -13.93 0.000 -.0321484 -.0242172 e29 | -.6082313 .002915 -208.66 0.000 -.6139446 -.6025181 e30 | -.144252 .0020102 -71.76 0.000 -.1481918 -.1403121 e31 | .0250983 .0022023 11.40 0.000 .0207819 .0294146 e32 | -.1352466 .0025166 -53.74 0.000 -.1401789 -.1303142 estrato1 | .189495 .0005579 339.69 0.000 .1884016 .1905883 estrato3 | -.322916 .0010167 -317.62 0.000 -.3249087 -.3209234 estrato4 | -.4792419 .0008096 -591.98 0.000 -.4808286 -.4776552 _cons | -1.493952 .0049766 -300.19 0.000 -1.503706 -1.484198 ------------------------------------------------------------------------------ Computing scores... Survey results: Survey: Poisson regression Number of strata = 4 Number of obs = 6089 Number of PSUs = 837 Population size = 27782772 Design df = 833 F( 51, 783) = 50.61 Prob > F = 0.0000 ------------------------------------------------------------------------------ | Linearized ncels | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- resmay6 | -.1189499 .0302713 -3.93 0.000 -.1783669 -.0595329 numradios | .0516967 .019889 2.60 0.010 .0126581 .0907352 nTVs | .1765135 .0210548 8.38 0.000 .1351867 .2178402 tfijo | -.1133193 .0389423 -2.91 0.004 -.1897558 -.0368828 tpaga | .1687625 .0330961 5.10 0.000 .1038009 .233724 luz | .760426 .4341483 1.75 0.080 -.0917272 1.612579 ncompus | .1401203 .0232643 6.02 0.000 .0944568 .1857837 internet | -.0047513 .0434178 -0.11 0.913 -.0899725 .0804699 prim2 | .1220794 .0353386 3.45 0.001 .0527162 .1914425 sec2 | .1243139 .0272578 4.56 0.000 .0708118 .177816 prepa2 | .0985697 .0253691 3.89 0.000 .0487748 .1483647 normal2 | -.0754923 .0678928 -1.11 0.266 -.2087533 .0577688 tec2 | .0036673 .0752994 0.05 0.961 -.1441316 .1514663 pro2 | .0751998 .038429 1.96 0.051 -.0002293 .1506288 m2 | .0292362 .04043 0.72 0.470 -.0501204 .1085929 doc2 | -.1606452 .0938375 -1.71 0.087 -.344831 .0235405 trabajadores | .1317087 .0193343 6.81 0.000 .093759 .1696583 e1 | .0016868 .1667689 0.01 0.992 -.3256499 .3290235 e2 | .0648902 .147677 0.44 0.660 -.2249727 .3547531 e4 | -.0181873 .1666332 -0.11 0.913 -.3452575 .3088829 e5 | -.1868012 .1583675 -1.18 0.239 -.4976475 .1240451 e6 | .1466339 .1557807 0.94 0.347 -.159135 .4524028 e7 | -.3935259 .2460116 -1.60 0.110 -.8764014 .0893497 e8 | -.1245186 .1549826 -0.80 0.422 -.4287209 .1796838 e9 | -.2299049 .1487182 -1.55 0.123 -.5218113 .0620015 e10 | -.083798 .2982254 -0.28 0.779 -.6691596 .5015637 e11 | -.3531031 .1536798 -2.30 0.022 -.6547482 -.0514579 e12 | -.4600155 .1721164 -2.67 0.008 -.7978484 -.1221826 e13 | -.3038536 .1661253 -1.83 0.068 -.629927 .0222199 e14 | -.1274274 .1614396 -0.79 0.430 -.4443035 .1894488 e15 | -.3696341 .151954 -2.43 0.015 -.6678918 -.0713763 e16 | -.0019455 .2189627 -0.01 0.993 -.4317289 .4278379 e17 | -.1287657 .1711779 -0.75 0.452 -.4647564 .207225 e18 | -.106628 .162161 -0.66 0.511 -.4249202 .2116641 e19 | -.1713332 .1540608 -1.11 0.266 -.4737261 .1310597 e20 | -.3297292 .247216 -1.33 0.183 -.8149688 .1555104 e21 | -.2838863 .1542873 -1.84 0.066 -.5867239 .0189512 e22 | -.079044 .1822687 -0.43 0.665 -.436804 .278716 e23 | .1517949 .1548563 0.98 0.327 -.1521596 .4557493 e24 | -.2631791 .179553 -1.47 0.143 -.6156086 .0892503 e25 | .0587084 .176449 0.33 0.739 -.2876286 .4050453 e26 | .0442957 .1611852 0.27 0.784 -.2720812 .3606725 e27 | .1651334 .168757 0.98 0.328 -.1661055 .4963723 e28 | -.0281828 .1619115 -0.17 0.862 -.3459853 .2896197 e29 | -.6082313 .1689908 -3.60 0.000 -.9399292 -.2765335 e30 | -.144252 .190234 -0.76 0.448 -.5176462 .2291423 e31 | .0250983 .1760391 0.14 0.887 -.3204341 .3706306 e32 | -.1352466 .2260741 -0.60 0.550 -.5789884 .3084952 estrato1 | .189495 .0642525 2.95 0.003 .0633791 .3156108 estrato3 | -.322916 .111296 -2.90 0.004 -.5413697 -.1044624 estrato4 | -.4792419 .1214264 -3.95 0.000 -.7175797 -.2409042 _cons | -1.493952 .462237 -3.23 0.001 -2.401238 -.5866655 ------------------------------------------------------------------------------ On Tue, Jul 27, 2010 at 12:02 PM, Stas Kolenikov <skolenik@gmail.com> wrote: > No, you don't have any problems with the degrees of freedom, which is > #PSUs - #strata = 837-4 = 833, and is reported as such. So I tend to > believe in Steven's story about empirical underidentification of the > overdispersion parameter: the likelihood is so flat in alpha that the > curvature (inverse of the variance) of the likelihood wrt this > parameter cannot be estimated with numeric accuracy that Stata would > find acceptable to report. And yes, this is an indication that > overdispersion is not such a great problem: coniditioning on > covariates and taking weights into account seems to make your data > approximately OK. > > As for the general convergence problems, they may be caused by the > scale of weights. Note that your log pseudo-likelihood has 8 digits > before the decimal point, and typically Stata wants to optimize things > down to 7 or so digits after the decimal point, that is, you need to > have about 15 reliable digits to declare convergence. That's too much > to ask for, as 15 digits is the accuracy limit of the -datatype- > double. In this situation (and in this situation only), it would be OK > to relax the convergence criteria by specifying something like > -ltolerance(1e-3)- instead of the default 1e-7; or rescale the weights > so that they sum up to say sample size rather than the population > size. > > On Tue, Jul 27, 2010 at 5:30 PM, Paolina Medina > <carmencitamedina@gmail.com> wrote: >> Thank you both, very much. >> So this almost zero alpha, without a confidence interval can be taken >> to indicate that there is no overdispersion in the model? >> Here is my svyset statement and the complete output.. >> I am using 52 regressors (including the constant), i really dont know >> how many are the design degrees of freedom... But in fact whenever i >> take any of these regressors i get a lot of troubles with convergence >> in the survey results (not concave or backed up) and i have to throw >> away many other regressors to get convergence again. >> Do you know anything i can do to fix this? > > -- > Stas Kolenikov, also found at http://stas.kolenikov.name > Small print: I use this email account for mailing lists only. > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > -- Paolina Medina Palma * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/