|
| From | Steven Samuels <sjhsamuels@earthlink.net> |
| To | statalist@hsphsun2.harvard.edu |
| Subject | Re: st: RE: Pearson chi square and Rao and Scott correction validity |
| Date | Fri, 7 Nov 2008 11:33:23 -0500 |
Ángel-Chamber and Skinner also suggest the model-based approach in Chapter 7. "Many hypotheses of interests may be expressed as a hypothesis in a log-linear model." It turns out that the Chi Square Pearson or Likelihood Ratio tests in an I x J table are equivalent to simultaneous tests that (I-!)(J-1) odds ratios are all equal to 1. In fact, a major competitor to the adjusted Chi square tests, with good properties in simulations, is doing Bonferroni adjusted tests of individual OR's.
I don't think that the logistic and multinomial logistic models are "cumbersome". The command set-up is quite simple( e.g. xi: -mlogit- y i.x ) The test of independence is just the test that the "x" coefficients in all equations are zero.
With the Chi-Square tests you are left only with a single "reject" or "cannot reject" answer. If the test rejects, then why? Modeling can give the answer. In the IxJ case, if there is any structure in either factor, you can assess models that are intermediate between complete saturation (IxJ parameters) and independence (I + J -1 parameters). And, you can ask whether other factors modify the degree of association. Stata can fit loglinear models not only with - mlogit- (a special case), but also with -glm- and -poisson-. In addition, the contributed commands -ipf- and -mclest- can be downloaded from SSC.
The original rules of thumb for Chi square tests were criteria for insuring that the test statistic indeed had a chi square distribution. Asymptotics depended on sample size. For the recommended tests in Skinner and Chamber's section 7.3.2, a major determinant of the accuracy of tests (level under the null hypothesis, power) is the number of clusters.
-Steve On Nov 7, 2008, at 8:35 AM, Ángel Rodríguez Laso wrote:
Reading back Korn and Graubard's (1999) Analysis of Health Surveys, Wiley, in page 78 they recommend for testing lack of association in a 2 x J contingency table a logistic regression with the presence/absence of the condition as the dependent variable, and for I x J contingency tables, a multinomial logit regression. I find this cumbersome. They consider chi-square statistics inappropriate in complex surveys, but they do not talk about the Rao and Scott correction.I will try to contact Korn or Graubard to see if they can add more light.Many thanks, Angel Rodriguez-Laso 2008/11/7 Steven Samuels <samplerx@earthlink.net>:I see that in my previous post I confused two issues 1) the sample size requirements for validity of the survey-adjusted chi square tests in Stata; 2) sample size requirements for estimates of cell totals or proportions, with small counts. Ángel asked about the first issue. Bottom line: I really don't have an answer. -Steve On Nov 6, 2008, at 5:04 PM, Steven Samuels wrote:I've looked though Chapters 6-7 of Chamber's and Skinner's book Analysis of Survey Data, Wiley, 2003, but I have no definitive answer. I do have some thoughts: * "Expected" count is not a guide in the survey setting--it is a sum of weights of sample observations in the table cell. * The accuracy of the second-order Rao-Scott statistic chi square, probably the best test in -svy: tab-, is apt to depend on the number of clusters, on the crude counts, and on the distribution of the observations across clusters. The rule of thumb of 5 observations (or 1) in a cell is based on theory of i.i.d. observations that does not hold in the complex survey setting. * With a small number of events, I ordinarily display only unweighted numbers and do not reported weighted estimates or confidence intervals. When I have wanted to infer something about a proportion based on small outcome count, I've resorted to the methods on pp. 64-68 of Korn and Graubard (1999) Analysis of Health Surveys, Wiley. A quick Google search turned up one survey which would not report a cell with fewer than 25 observations (http://www.nsf.gov/statistics/ showsrvy.cfm?srvy_CatID=5&srvy_Seri=16) and another in which the minimum cell size was 4,000! (http://www.phac-aspc.gc.ca/publicat/ cdic-mcc/17-3/a_e.html). So a guess for Ángel is that not even five observations in table cell is enough. -Steve On Nov 6, 2008, at 7:33 AM, Nick Cox wrote:There is no need to invoke belief! My -tabchi- and -tabchii- (programs) from the -tab_chi- package on SSC do indeed give warnings. (There is no Stata program called tab-chi.) But these old warnings are very conservative. Many writers now advise that chi-square works fine so long as all expected frequencies are above about 1. In any case, the point can be explored by simulations or bootstrapping. Often it is better to use Fisher's exact test. I can't advise on the main issue, which is for svy-savvy people, but in general very low expected frequencies could be problematic for any method. Nick n.j.cox@durham.ac.uk Ángel Rodríguez Laso I've been reviewing the manuals and statalist archives and I'veconfirmed that Stata does not give any automatic warning message whenrequirements for a valid chi-square test are not met (i.e. no more than 20% of the expected values in a table are less than 5 and none are less than 1), what I think is a nuisance. I suppose this can be only worked out by writing the option 'expected' after tabulate andchecking oneself if the requirements are met. I believe Cox's tab-chipackage does give a warning when requirements are not met. I wonder also if the Rao and Scott correction of Pearson chi-square that is recommended for survey designs needs the same requirements. The problem then would be that -svy:tab- doesn't support the 'expected' option neither tab-chi is suitable for survey analysis. * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/Steven Samuels 845-246-0774 18 Cantine's Island Saugerties, NY 12477 EFax: 208-498-7441 * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/* * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/
* * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/
| © Copyright 1996–2025 StataCorp LLC | Terms of use | Privacy | Contact us | What's new | Site index |