Re: st: RE: one-tailed tests

[Roger Newson <r.newson@imperial.ac.uk>]

My point was that tests are a means to the end of defining confidence 
regions. These confidence regions may be confidence intervals for a 
scalar parameter (eg a difference between 2 probabilities of voting 
Democrat), or k-dimensional confidence regions for a k-vector parameter 
(eg the Bonferroni boxes recommended for Bea Potter's parameter 
(b1,b2)), or even power-set-valued confidence regions for a set-valued 
parameter (eg "the set of null hypotheses that are true", estimated 
using multiple-test procedures).

The confidence region is defined (at least approximately) as the set of 
parameter values that pass the test. Most of these applications work 
best when the tests are 2-tailed. In the case of confidence intervals, 
the test must usually be 2-tailed for the confidence interval to have 
both an upper limit and a lower limit. In the case of multiple-test 
procedures, it helps if the underlying tests are 2-tailed, because, that 
way, the P-values are non-negatively correlated (even if the test 
statistics may be negatively correlated), and this makes the 
multiple-test procedure less conservative. Occasional exceptions arise 
when the distribution of the test statistic, under some hypotheses, 
really is one-tailed. This typically leads to confidence intervals 
bounded only on one side (eg confidence intervals for odds ratios 
extending from a lower limit to infinity), or to a requirement for 
conservative multiple-test procedures (eg the Holm procedure).

I hope this helps.

Best wishes

Roger


Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: r.newson@imperial.ac.uk
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/

Opinions expressed are those of the author, not of the institution.

On 08/07/2010 15:01, Nick Cox wrote:
> I don't know why anyone would want to do any test if you can get a
> confidence interval. After all, most of the best science should be
> quantitative, and a confidence interval is a report on the measurement
> scale of interest: it summarizes information, it indicates uncertainty
> and it can be compared with your qualitative ideas on sign or direction
> quite easily. If you think that something should be positive, or
> increase, or whatever, you can see whether the c.i. lies the correct
> side of zero, or whatever. In contrast, a yes-no test decision or even a
> P-value is a more cryptic and problematic reduction of the data.
>
> Actually, I don't know why anyone would get a confidence interval if you
> can put most or all of the data on a graph. After all, a good graph
> shows the main features of the data and detailed departures from those
> features, and it can give you ideas on what next to do.
>
> I have to estimate the extent to which Eric or Roger was talking tongue
> in cheek, and you will have to do the same about this.
>
> Nick
> n.j.cox@durham.ac.uk
>
> Eric Uslaner
>
> Bea Potter asked how to do one-tailed tests and Roger Newson responded:
> "I don't know why anybody would want to do a 1-tailed test, except if
> the
> distribution of the test statistic, under the null hypothesis, really IS
> one-tailed."
>
> I don't understand why anyone would do a two-tailed test.  If you have a
> theory, that theory is directional: The more A, the more B.  A
> two-tailed test is completely inappropriate for this.  E.g., in studies
> of voting behavior in political science, we usually posit that party
> identification is the dominant force behind voter choice.  So if you are
> a strong Democrat, you will vote Democratic.  A two-tailed test would
> imply that a strong Republican would be equally likely as a strong
> Democrat to vote Democratic.  This is plain silly.  If you don't have a
> theoretical focus, you don't have a research design worth submitting
> anywhere.
>
> So I don't understand why anyone does two-tailed tests and why Stata
> (together with other statistical programs) report them as defaults.
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/