Re: st: Bootstrapping question

["JVerkuilen (Gmail)" <jvverkuilen@gmail.com>]

On Fri, Feb 8, 2013 at 9:49 AM, Austin Nichols <austinnichols@gmail.com> wrote:
> Bootstrapping is designed to improve CIs by making their coverage
> better, not by making them smaller!

Yes, in many circumstances they are markedly wider.



A better approach for your case
> would be Bayesian, but again a better CI does not necessarily mean a
> smaller CI. Some priors might result in smaller credible intervals,
> but others in larger, and you need to describe the dependence of your
> results on your assumptions. If you estimate 15 ways and only report
> the one that accords with the conclusion you want, you are committing
> scientific fraud.

Yes, this is a real issue and scholars often don't do a good job
describing their specification search, of which this is definitely an
example.

I'm in the process of writing a paper on the small sample estimation
of ICC and reliability coefficients (which psychometricians frequently
care about). Nearly all the work out there is on method of moments
unbiased estimators and that's what people expect. They're not very
good of course but if you're expecting it and someone gives you
something else it's necessary to have some explanations.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/