Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | Stas Kolenikov <skolenik@gmail.com> |
| To | statalist@hsphsun2.harvard.edu |
| Subject | Re: st: Spearman correlations with survey data |
| Date | Thu, 8 Nov 2012 12:20:02 -0600 |
To add to Roger's response: as a survey statistician, I want to work with the stuff that has clear finite population definitions, and clear sample analogues that would estimate the population target. I do understand what the population parameter for Kendall's tau is, and hence how to get to that parameter with approximate survey inference, as indicated by Roger's -somersd- line of code. However, I don't understand what the population parameter is for Spearman correlation, so it is not at all clear what it is going to be that -svy : spearman- would do if it worked (thankfully, it does not). Conceptually, one can say that there is the census Spearman correlation for the finite population, but whether the sample Spearman correlation is any good in estimating it, I don't know (and doubt that); likewise, I don't know what a weighted rank is, or should be. With a long stretch of finite population inference imagination, you can argue that Spearman correlation is a U-statistic of order 2, so estimating it properly requires the second order selection probabilities, and estimating its variance, the fourth order selection probabilities. That's a long-winded way to say, "this is not doable". -- -- Stas Kolenikov, PhD, PStat (SSC) :: http://stas.kolenikov.name -- Senior Survey Statistician, Abt SRBI :: work email kolenikovs at srbi dot com -- Opinions stated in this email are mine only, and do not reflect the position of my employer On Thu, Nov 8, 2012 at 12:00 PM, Roger B. Newson <r.newson@imperial.ac.uk> wrote: > I would second that "Amen". And I would recommend the use of Kendall's tau-a > as an alternative, because (a) the Central Limit Theorem works a lot faster > for Kendall's tau-a than for Spearman's rho and (b) Kendall's tau-a is > interpretable (in words) as a difference between 2 probabilities, namely a > probability of concordance and a probability of discordance. I find this > difference between probabilities easier to understand than a measure of rank > linearity (which is what Spearman's rho is), and also more useful to know, > as I would prefer a good non-linear monotonic predictor to a second-rate > linear predictor. Spearman's rho, by contrast, is MUCH easier to estimate > without a computer, which was an important issue before we had computers. > > To estimate Kendall's tau-a with confidence limits in Stata with clustering > and/or sampling-probability weighting, use the -somersd- package, > downloadable from SSC. As in: > > somersd x y [pwei=mysampwt], taua tdist transf(z) cluster(mypsu) > > I have not yet introduced confidence intervals allowing for sampling strata. > However, sampling-probability weights and clustering are available. And the > -somersd- package comes with .pdf manuals, and with hypertext references in > the on-line help to further documentation of the methiods and formulas. > > I hope this helps. > > Best wishes > > Roger > > > Roger B Newson BSc MSc DPhil > Lecturer in Medical Statistics > Respiratory Epidemiology and Public Health Group > National Heart and Lung Institute > Imperial College London > Royal Brompton Campus > Room 33, Emmanuel Kaye Building > 1B Manresa Road > London SW3 6LR > UNITED KINGDOM > Tel: +44 (0)20 7352 8121 ext 3381 > Fax: +44 (0)20 7351 8322 > Email: r.newson@imperial.ac.uk > Web page: http://www.imperial.ac.uk/nhli/r.newson/ > Departmental Web page: > http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/ > > Opinions expressed are those of the author, not of the institution. > > > On 08/11/2012 17:26, Lachenbruch, Peter wrote: >> >> Amen! In fact, tests on Spearman coefficients are notoriously sensitive >> to normality. An article by Egon Pearson in Biometrika in the 1970s showed >> this clearly. Sorry i don't have the reference at hand. >> >> Peter A. Lachenbruch, >> Professor (retired) >> ________________________________________ >> From: owner-statalist@hsphsun2.harvard.edu >> [owner-statalist@hsphsun2.harvard.edu] on behalf of Nick Cox >> [njcoxstata@gmail.com] >> Sent: Thursday, November 08, 2012 1:58 AM >> To: statalist@hsphsun2.harvard.edu >> Subject: Re: st: Spearman correlations with survey data >> >> Spearman correlation is just Pearson correlation applied to ranks, so >> ranking first (use -egen-) gets you from one to the other. Otherwise >> P-values for correlations are over-rated in my view, whether in -svy- >> contexts or otherwise. >> >> Others should have comments on the -svy- aspects. >> >> Nick >> >> On Thu, Nov 8, 2012 at 5:42 AM, Lee Grenon <lgrenon@sfu.ca> wrote: >> >>> I am interested in calculating Spearman correlations for complex survey >>> data. As I understand, I can calculate Pearson correlations using corr with >>> aweight for the coefficients and then calculate the p-values using svy: >>> regress y x and svy: regress x y then selecting the larger p-value. Is there >>> a way of calculating Spearman correlations using a survey weight and >>> bootstrap weights? >> >> >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/faqs/resources/statalist-faq/ >> * http://www.ats.ucla.edu/stat/stata/ >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/faqs/resources/statalist-faq/ >> * http://www.ats.ucla.edu/stat/stata/ >> > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/faqs/resources/statalist-faq/ > * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/