st: Re: statalist-digest V4 #4814

[Dirk Enzmann <dirk.enzmann@uni-hamburg.de>]

David,

In principle, I agree.

Which reminds me to the old Radio Yerevan jokes, which always started 
with "Is it true that ...?" to which the answer was "In principle yes, 
but ..."

My "but" in this case would be that examining overlap to examine 
significance is possible if you do not apply the too simplistic "no 
overlap" rule. This I tried to stress because I think that it is very 
useful to add confidence intervals to graphic displays of mean 
differences (also to avoid a substantive interpretation of non 
significant differences).

To which, of course, the without any doubt correct reaction will be "But 
if exact p values are desired they should also be presented."

And: You are right that readers could misinterpret my quote from 
Schenker and Genlemen (2001). Thanks for clarifying.

Dirk

Wed, 6 Mar 2013 20:58:55 -0500 David Hoaglin<dchoaglin@gmail.com> wrote:

> Subject: Re: st: confidence intervals overlap [was: positive interaction - negative covariance]
>
> Dirk,
>
> I don't agree that that warning goes too far.  It is consistent with
> the advice given by both Schenker and Gentlemen (2001) and Cumming and
> Finch (2005).
>
> Readers who are not familiar with the paper by Schenker and Gentleman
> might interpret the material that you quoted as an endorsement of
> judging significance according to overlap.  In fact, the quoted
> material appears at the beginning of the first paragraph of the
> introduction.  That paragraph concludes as follows: "Over a number of
> years, however, we have tried to discourage the practice of examining
> overlap to assess significance among authors of articles that we have
> reviewed, because the procedure can lead to mistaken conclusions."
>
> The paper by Cumming and Finch is highly useful.  I'm glad you called
> attention to it.  As they make clear, their "Rules of Eye" are
> intended to help readers when an article includes, for example, a
> figure that shows confidence intervals for the means of two
> independent samples and does not show a confidence interval for the
> difference between those means.  On page 174 they say, "By analogy
> with rules of thumb, our rules are intended to be useful heuristics or
> pragmatic guidelines,  They are not intended to be numerically exact
> or to replace statistical calculations: If exact p values are desired,
> they should also be presented."
>
> David Hoaglin

========================================
Dr. Dirk Enzmann
Institute of Criminal Sciences
Dept. of Criminology
Rothenbaumchaussee 33
D-20148 Hamburg
Germany

phone: +49-(0)40-42838.7498 (office)
       +49-(0)40-42838.4591 (Mrs Billon)
fax:   +49-(0)40-42838.2344
email: dirk.enzmann@uni-hamburg.de
http://www2.jura.uni-hamburg.de/instkrim/kriminologie/Mitarbeiter/Enzmann/Enzmann.html
========================================
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/