st: Re: st: Jackknife standard e​rrors using replicat​e weights in Stata 1​2.1​

[William Buchanan <william@williambuchanan.net>]

Hi Annelies,

I think we would need to see more information regarding the differences between the output.  I did some work with a national education dataset here in the US and was able to replicate the results that ED published from AM (designed by American Institutes for Research for working with NAEP data), WesVar, and SUDAAN.  The documentation wasn't the clearest to me at the time, but I was able to get identical results (down to the thousandths/ten-thousandths).  Did the sampling design involve multistage sampling?  Are the probability weights that you specified correct for your analytical purposes (e.g., they wouldn't be the school probability weights if you're trying to estimate individual level relationships).  

The data set I worked with had jackknife replicate weights and also had probability weights that were intended when using Taylor series SE.  

HTH,
Billy





On Feb 19, 2013, at 7:36 AM, Annelies Blom <blom@survex.de> wrote:

> Dear colleagues,
> 
> for some analyses that I am currently conducting I have a dataset with a complex
> sample design with replicate weights. The replicate weights were produced with a
> jackknife(1) procedure. Various analyses are to be conducted with this setup.
> Other project partners conduct the analyses in WesVar and our results have to be
> comparable to theirs. I  prefer using Stata, but am encoutering problems.
> 
> An example of my svyset and analysis command:
> svyset [pw = weight0], jkrw(weight1 - weight80, multiplier(1))
> svy jackknife: proportion age
> 
> The following setup and command yields the same results:
> svyset [pw = weight0], jkrw(weight1 - weight80, multiplier(1)) vce(jackknife)
> svy: proportion age
> 
> The problem:
> The standard errors calculated by Stata with these commands are unplausibly
> small. In addition, for some variables Stata does not give any SEs at all (or no
> confidence intervalls). The SEs when not specifying jackknife SEs (i.e. when
> calculating Taylor linearized SEs, which is the default) are much more
> plausible. And they are complete.
> 
> Via the Stata search function I found
> <http://www.ats.ucla.edu/stat/stata/library/replicate_weights.htm>
> These researchers state that "although the command will run (and run faster)
> without the jackknife option after the svy, you will get linearized standard
> errors instead of the jackknife standard error.  This jackknife standard error
> matches the standard errors produced by both SUDAAN and WesVar."
> They refer to Stata 9, while I am working in Stata 12.1. According to them, if I
> do not specify jackknife SEs, my results are not be comparable to the WesVar
> results.
> 
> My questions:
> - Are the jackknife SEs calculated by Stata 12.1 correct? If so, why do I get
> missing SEs?
> - How can I solve these problems while at the same time remaining comparable
> with analyses conducted in WesVar?
> 
> With best wishes,
> Annelies
> 
> ---
> Annelies Blom, Ph.D.
> Survex - Survey Methods Consulting
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/