st: RE: How easy to get caught by rounding error

["Keith Dear" <Keith.Dear@anu.edu.au>]

Yes that is worrying:

. di %20.16f 7*180/10
126.0000000000000000

. di %20.16f 7/10*180
125.9999999999999900

. di .7*180
126

. di floor(.7*180)	// as per -help binomialtail-
125

Can't be too careful it seems! 
But I have to wonder WHY the function uses floor() -- it seeems to lay a
trap without adding any useful functionality.
Safer would be to issue an error if the supposedly integer arguments
aren't. Is there a major downside to this?
Anyway, thanks for the warning.
kd



-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Allan Reese
(Cefas)
Sent: Tuesday, 14 December 2010 4:29 AM
To: statalist@hsphsun2.harvard.edu
Cc: David Smith (Cefas); David Maxwell (Cefas); Jon Barry (Cefas); Mark
Thrush (Cefas); Nick Taylor (Cefas)
Subject: st: How easy to get caught by rounding error

Tabulating some binomial tail probabilities, I was startled that they
were not monotonic.  To cut a long story short, the erroneous result was


binomialtail( 180 , .7*180 , .75), which was different from
binomialtail( 180 , 126 , .75), 
although .7*180 insisted on printing as 126 

binomialtail( 180 , 180/10*7 , .75) did equal binomialtail( 180, 126 ,
.75) as did binomialtail( 180, round(.7*180) , .75) 

But 
binomialtail( 180 , .7/180 , .75) = binomialtail( 180 , 125 , .75)

It's made me think how hard it is to *prove* that a computer program is
correct.  round() should fix this one - for all values?

Allan


R Allan Reese
Senior statistician, Cefas
The Nothe, Weymouth DT4 8UB 

Tel: +44 (0)1305 206614 -direct
Fax: +44 (0)1305 206601 

www.cefas.co.uk 




************************************************************************
***********
This email and any attachments are intended for the named recipient
only.  Its unauthorised use, distribution, disclosure, storage or
copying is not permitted.  If you have received it in error, please
destroy all copies and notify the sender.  In messages of a non-business
nature, the views and opinions expressed are the author's own and do not
necessarily reflect those of the organisation from which it is sent.
All emails may be subject to monitoring.
************************************************************************
***********


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/