Stas - Thanks for your further comments and the reference. With
-somersd- and the fun(), cluster(), and wstrata() options I can get
pairwise estimates of the "correct" covriance matrix obtained by
jackknife, as I described earlier - but it turns out that (at least for
the one time I tried it) the 3 x 3 matrix is singular! So to do the
jackknife on all three taus or on the partial tau, it looks as though I
would need to modify the current -somersd-, Another alternative is to
use -bootstrap- on -somersd- with the appropriate options, but this is
wasteful since I only need the point estimates for each resampled data
set, not jackknife-within-jackknife standard errors. But the advantage
of bootstrapping -somersd- is that I will have the necessary informaiton
to compare the two methods as you suggest. Of course there is also the
issue of which transformation to use. It was also interesting to read
your discussion about what is "asympotic". The application I have in
mind is more like #strata going to infinity with #units per stratum
bounded.
By the way, I just now realized what you were talking about with respect
to "1, 3, 4" oops!
Al
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Stas
Kolenikov
Sent: Wednesday, January 21, 2009 3:17 PM
To: [email protected]
Subject: Re: st: Partial Correlation Kendall's Tau (or Somers D)
Al,
I was picking on you posting items 1, 3 and 4 in the original email :))
Looks like you could do it both ways, then. You can get the covariance
matrix of the taus, provided that it accounts for the complex sample
design the way you need. Working with U-statistics in stratified samples
is a weird thing to do. I think the general asymptotic normality in
complex survey designs was only proven in the 1990s (I cannot find a
proper reference, but here's a partial older answer --
http://www.citeulike.org/user/ctacmo/article/1743121). Whether
-wstrata- or -bstrata- options of -somersd- should be used, or yet
something else that would need to involve all units in a proper
Horvitz-Thompson type estimator -- that is your call as the analyst.
Keep in mind that there is no single asymptotics in survey statistics,
too. The most popular setups corresponding to the most popular designs
with 2 PSU/stratum is to have # strata \to \infty with # units/stratum
bounded. Those are the more complex settings to analyze compared with
fixed # strata and growing # units per stratum asymptotics.
I wouldn't say that the delta method is always preferrable to the
jackknife, or the other way round. The general theory is that in most
survey settings, they are equivalent to the second order (i.e., agree to
the terms of $O(n^{-2})$) while the bootstrap (provided it is done
right) agrees with them to the order of $O(n^{-1})$, and BRR agrees to
the order of $O(n^{-1/2})$. What I would probably do is to code it both
ways, to make sure they produce answers that match to the fourth decimal
point or so.
On 1/21/09, Feiveson, Alan H. (JSC-SK311) <[email protected]>
wrote:
> Stas - Thanks for pointing out the asymptotic joint normality property
> for U-statistics. Your comments suggest that if I could get my hands
> on the standard error covariance matrix for the original tau-s, I
> could try using the delta method to get a standard error for
tau_xy.z.
>
> I think I can get this by three runs of -somersd-.
>
> 1.
> somersd y x z
> matrix Vyxz = e(V)
> matrix list Vyxz
> symmetric Vyxz[3,3]
> y x z
> y 0
> x 0 .11
> z 0 -.09777778 .09
>
> So Var(tau_yx) = 0.11, Cov(tau_yx, tau_yz) = -.09777778, Var(tau_yz)
> =0.09
>
>
> 2.
>
> somersd x y z
> matrix Vxyz = e(V)
> ....
> gives Var(tau_xy), Cov(tau_xy, tau_xz), etc
>
> 3. somersd z x y
> ...
> gives Cov(tau_zx, tau_zy),etc
>
> So if this is correct, the problem can be approached by only counting
> to three - not four!
>
> Of course, the delta method on the original tau-s may not be as good
> as doing jackknife on resampled tau_yx.z values - but the former is
> easy to implemnet given -somersd- in its present form.
>
>
> Al
>
>
>
>
>
> It looks as thoiug I could do this by running somersd twice, once
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Stas
> Kolenikov
> Sent: Wednesday, January 21, 2009 8:29 AM
> To: [email protected]
> Subject: Re: st: Partial Correlation Kendall's Tau (or Somers D)
>
> Al,
>
> there are three types of statisticians: those who can count to four
> and those who cannot :)).
>
> Since the original tau's are U-statistics, they will have a (joint
> multivariate) asymptotic normal distribution, and hence the
> partialized version you presented would also be asymptotically
normal.
> The necessary condition for the jackknife standard errors to be
> consistent is that the statistic of interest has an asymptotically
> normal distribution, and I would guess that other more subtle
> regularity conditions would also be satisifed (although jackknife is
> not consistent say for a median which is also asymptotically normal).
> If you have a complex survey design then you would need to omit the
> complete PSU when computing the standard errors, and Stata's -svy
> jackknife- does that for you (although you would need to write a
> wrapper of -eclass, properties(svyj)- and see that the conditions
> outlined for those properties are satisfied).
>
> You guys do have some powerful computers at NASA, I guess. I wouldn't
> think of doing jackknife over -ktau- with the capacities I have :)).
>
> On 1/21/09, Feiveson, Alan H. (JSC-SK311) <[email protected]>
> wrote:
> > Hi - I have been reading in Gibbons and Chakraborti (Nonparametric
> > Statistical Inference) about a Kendall's Tau analog to partial >
> correlation, where one would like to quantify the association between
> > y and x after correcting for z. Specifically, the authors define >
> tau_xy.z in terms of the three pairwise associations tau_xy, tau_yz,
> > and tau_xz, and then give an expression that loooks exaclty like >
> Pearson partial > correlation:
> >
> >
> > tau_xy.z = (tau_xy -
> tau_xz*tau_yz)/sqrt((1-tau_xz^2)*(1-tau_yz^2))
> >
> >
> > My questions on this are
> >
> > 1. Can the jackknife method for standard errors be extended to >
> tau_xy.z or its Somers D analog?
> >
> > 3. Are there extensions to defining tau_xy.z within or betwen
> strata > and for obtaining standard errors with clusters as Roger
> Newson has done?
> >
> > 4. Most importantly - Roger - do you have any plans for updating
> your
>
> > Somers D program to include partial association?
> >
> > Thanks,
> >
> > Al Feiveson
> >
>
> --
> Stas Kolenikov, also found at http://stas.kolenikov.name Small print:
> I
--
Stas Kolenikov, also found at http://stas.kolenikov.name Small print: I
use this email account for mailing lists only.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
