st: RE: RE: How easy to get caught by rounding error

[Nick Cox <n.j.cox@durham.ac.uk>]

Not disagreeing, just amplifying slightly: 

The division takes place first and 0.7 does not have an exact binary equivalent. 

Although a little thought shows that the calculation reduces to 7 * 18 and should yield an integer, you get what you asked for in either case, Stata's best stab at what you asked it to calculate. 

Nick 
n.j.cox@durham.ac.uk 

Keith Dear

Yes that is worrying:

. di %20.16f 7*180/10
126.0000000000000000

. di %20.16f 7/10*180
125.9999999999999900

. di .7*180
126

. di floor(.7*180)	// as per -help binomialtail-
125

Can't be too careful it seems! 
But I have to wonder WHY the function uses floor() -- it seeems to lay a
trap without adding any useful functionality.
Safer would be to issue an error if the supposedly integer arguments
aren't. Is there a major downside to this?
Anyway, thanks for the warning.

Allan Reese

Tabulating some binomial tail probabilities, I was startled that they
were not monotonic.  To cut a long story short, the erroneous result was


binomialtail( 180 , .7*180 , .75), which was different from
binomialtail( 180 , 126 , .75), 
although .7*180 insisted on printing as 126 

binomialtail( 180 , 180/10*7 , .75) did equal binomialtail( 180, 126 ,
.75) as did binomialtail( 180, round(.7*180) , .75) 

But 
binomialtail( 180 , .7/180 , .75) = binomialtail( 180 , 125 , .75)

It's made me think how hard it is to *prove* that a computer program is
correct.  round() should fix this one - for all values?


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/