RE: st: Question about floating point precision

["Nick Cox" <n.j.cox@durham.ac.uk>]

Phil's good answer can be supplemented. 

The underlying principle, as Markus knows well, is that what look like
exact decimals don't necessarily correspond to exact binaries. In fact,
sometimes you get exactly the right result, sometimes a smidgen too
much, and sometimes a smidgen too little.  

In this particular case, let's follow the point made clear by Phil. The
format %21x lets you drill down a little deeper to see what is
happening. 

. di %21x  26.07
+1.a11eb851eb852X+004

. di %21x  52.14
+1.a11eb851eb852X+005

shows that luck, or caprice, or the numbers, can be on Markus' side.
52.14 happens to be held internally as _exactly_ twice what 26.07 is
held as, so the ratio comes out as 0.5 and Markus gets exactly the right
answer in his first example. That's good fortune not matched in the
other ways of holding numbers. 

Nick 
n.j.cox@durham.ac.uk 

Phil Schumm

On Mar 30, 2010, at 12:37 AM, Markus Hahn wrote:
> ----------------------------------
> . // why is this code more precise
> .
> . di %22.18f 1559*26.07/52.14
> 779.500000000000000000
>
> .
> . // than this code
> .
> . di %22.18f 1559*26.07
> 4.064313000000000e+04
>
> . di %22.18f 40643.13/52.14
> 779.499999999999890000
> ----------------------------------
>
> Why is the code in the first part of my example more precise than  
> the last part? Is it really more precise or does it only SEEM to be  
> so? What I would like to know is whether the first part is more  
> precise (if at all) because it only uses one expression for the  
> calculation as compared to last part which uses two expressions?  
> Does the indirect calculation introduce this imprecision?


The difference is due to the order of operations -- remember, / comes  
before *.  To wit:

. di %21x 1559*26.07/52.14
+1.85c0000000000X+009

. di %21x (1559*26.07)/52.14
+1.85bffffffffffX+009

. di %21x 40643.13/52.14
+1.85bffffffffffX+009


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/