st: How easy to get caught by rounding error

["Allan Reese (Cefas)" <allan.reese@cefas.co.uk>]

Tabulating some binomial tail probabilities, I was startled that they
were not monotonic.  To cut a long story short, the erroneous result was


binomialtail( 180 , .7*180 , .75), which was different from
binomialtail( 180 , 126 , .75), 
although .7*180 insisted on printing as 126 

binomialtail( 180 , 180/10*7 , .75) did equal binomialtail( 180, 126 ,
.75) as did binomialtail( 180, round(.7*180) , .75) 

But 
binomialtail( 180 , .7/180 , .75) = binomialtail( 180 , 125 , .75)

It's made me think how hard it is to *prove* that a computer program is
correct.  round() should fix this one - for all values?

Allan


R Allan Reese
Senior statistician, Cefas
The Nothe, Weymouth DT4 8UB 

Tel: +44 (0)1305 206614 -direct
Fax: +44 (0)1305 206601 

www.cefas.co.uk 




***********************************************************************************
This email and any attachments are intended for the named recipient only.  Its unauthorised use, distribution, disclosure, storage or copying is not permitted.  If you have received it in error, please destroy all copies and notify the sender.  In messages of a non-business nature, the views and opinions expressed are the author's own and do not necessarily reflect those of the organisation from which it is sent.  All emails may be subject to monitoring.
***********************************************************************************


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/