st: RE: RE: RE: RE: RE: A modest proposal - missing data doesn't count

["Nick Cox" <n.j.cox@durham.ac.uk>]

Allan's Stata use goes back long before these functions. He was
publishing on Stata in 1991. 

. search reese, author historical

STB-17  cc2 . . . . . . . . . . . . . Running the Stata tutorials on a
network
        . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.
Reese
        1/94    pp.3--4; STB Reprints Vol 3, pp.14--15           (no
commands)

STB-3   os1.1 . . . . . . . . . . .  Update on gphpen and color
postscript use
        . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  R. A.
Reese
        9/91    pp.10--11; STB Reprints Vol 1, p.55
        Color postscript support added in Stata 3.0

STB-1   os1 . . . . . . . . . . . . . . . . . . .  gphpen and color
postscript
        . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  R. A.
Reese
        5/91    p.10; STB Reprints Vol 1, p.54
        Color postscript support added in Stata 3.0

He has about 20 years' experience of being bitten by this -- and I guess
has not taken to newer "solutions". 

Nick 
n.j.cox@durham.ac.uk 

Martin Weiss

Exactly, that is precisely why I always add a -& !mi()- to my
comparisons.
Or use -inrange()-, as you suggested earlier.

***********
sysuse auto, clear
cou if rep78>3
cou if rep78>3 & !mi(rep78)
cou if inrange(rep78,4,.)
***********

So still: What is the fuss about?

Nick Cox

The fuss is that 

if x > 10 

includes x >= . Numeric missings all count as larger than any
non-missing numeric value. 

Nick 
n.j.cox@durham.ac.uk 

Martin Weiss

I got used to the -!mi()- thing in "bigger-than" comparisons after a
couple
of errors in this area, and I do not even regard it as a hassle anymore.
What is all the fuss about?

Nick Cox

This comes up on the list, understandably, about once a year. 

I think everybody has some, indeed much, sympathy with the problem, but
no-one ever likes anybody else's solution. 

To make even a case for his new function, Allan has got to show that it
gives sensible answers in all situations in which it might be applied,
and not just inequalities. Just as it's a feature that the name
-value()- is not bespoke, it is also a weakness that it carries no
meaning that is utterly evident and precise. For example, is value(.)
the same or different from value(.a)? Does value(x) < 10 include or
exclude missings? If they're excluded too, then Allan wants a three-way
logic in Stata, and the quickest way to get that is to write your own
clone of Stata. (Allow 25 years.) 

(If this isn't Phil Schumm's point, it's very close.) 

-inrange()- is a well-defined function available for comparisons. I
don't think many people use it, perhaps because its detailed rules don't
stick in one's memory. By the time most people have looked up the syntax
they could write out a longer expression avoiding it. 

I think -value()- could hardly jump such a hurdle. 

Nick 
n.j.cox@durham.ac.uk 

Allan Reese

Feeling aggrieved at years of typing expressions like "if x>10 & x<.",
and feeling sure there must be lots of wrong results from Stata because
users didn't notice that "x>10" includes cases with the missing value,
might I request a new code function?  

The name value() is available. While users would have to remember to use
it, it would quickly become a habit to type "if value(x)>10" etc, with
the effect that the expression would evaluate as true only when x had a
non-missing value.  A check box could be added to the menu interface as
a reminder.    

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/