Hi,
I'm runnig a fixed-effects and random-effects logit with DHS
(Demographic Health Survey) data. The groups are the clusters within
which each female belongs to. Given the recomendation in a previous
statalist mail (http://www.stata.com/statalist/archive/2007-06/msg00818.html
), I calculated the percentege correctly predicted for both models.
Nevertheless, I'm very surprised because the sensitivity (% correctly
predicted of positive outcomes) is very low for the fixed-effects
logit while the specificity (% correctly predicted of negative
outcomes: zeros) very high. On the other hand, both measures are more
acceptable for the random effects models. Besides, the pvalue of the
hausman test is zero.
Are this measures (sensititvity and specificity) the most appropiate
for measuring the quality of the results of this kind of models? And
are the results I've gotten frequent when comparing a fixed effects
versus a random effects model?
Here are the results given by rocss, a command the calculates the
sensitivity (sens) and specificity (spec) for different cutoffs:
Fixed-effects logit:
cutoff sens spec omspec cclass carea
------------------------------------------------------
1. 0.000 1.0000 0.0000 1.0000 47.5149 0.0000
2. 0.100 0.7250 0.6656 0.3344 69.3837 0.5741
3. 0.200 0.4250 0.9026 0.0974 67.5660 0.7104
4. 0.300 0.2313 0.9637 0.0363 61.5734 0.7304
5. 0.400 0.1219 0.9908 0.0092 57.7961 0.7352
6. 0.500 0.0580 0.9951 0.0049 54.9844 0.7356
7. 0.600 0.0209 0.9995 0.0005 53.4507 0.7358
8. 0.700 0.0078 1.0000 0.0000 52.8543 0.7358
9. 0.800 0.0036 1.0000 0.0000 52.6555 0.7358
10. 0.900 0.0006 1.0000 0.0000 52.5135 0.7358
11. 1.000 0.0000 1.0000 0.0000 52.4851 0.7358
+------------------------------------------------------+
Random-effects logit:
cutoff sens spec omspec cclass carea
------------------------------------------------------
1. 0.000 1.0000 0.0000 1.0000 48.9190 0.0000
2. 0.100 0.9791 0.1930 0.8070 57.7572 0.1910
3. 0.200 0.9291 0.3791 0.6209 64.8135 0.3685
4. 0.300 0.8490 0.5321 0.4679 68.7099 0.5046
5. 0.400 0.7528 0.6726 0.3274 71.1808 0.6171
6. 0.500 0.6353 0.7809 0.2191 70.9670 0.6923
7. 0.600 0.5036 0.8721 0.1279 69.1851 0.7442
8. 0.700 0.3871 0.9293 0.0707 66.4053 0.7697
9. 0.800 0.2448 0.9674 0.0326 61.3923 0.7817
10. 0.900 0.0971 0.9930 0.0070 55.4764 0.7861
11. 1.000 0.0000 1.0000 0.0000 51.0810 0.7864
Any help would be very appreciated.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
