Re: whole number constants

dancerchris@[EMAIL PROTECTED]
 wrote:
(snip)

> !  this looks good/ledgable but.....

> x = 2*pi
> y = x/2

> !  does the following result...

> if (y == pi) then
> !   aha cancellation
> else
> !   integer value of two assigned different values of 'extra' digits
> !   (beyond precision of dp) ergo not a constant
> end if

For REAL x and y of any kind, the result should be the same
for INTEGER 2, or REAL 2 of the appropriate kind.  That, however,
doesn't guarantee equality.  It should be true that multiplying
a binary floating point value by two leaves the significand
the same, but there might even be exceptions.
(Some Cray processors being a possible exception.)
  It isn't necessarily true for non-binary floating point,
including base 10 and 16.

Consider instead multiplying the value of pi/4 by two
using decimal or base 16 floating point.  The product is
greater than one, which will ****ft the significand right
one decimal or hexadecimal digit during post normalization.

A consequence is that (x+y)/2, when x and y are equal,
may not equal x and y.  The solution, used in the
final iteration for some square root algorithms,
is to use x+(y-x)/2.

There are also effects due to systems keeping more precision
than one would expect through a calculation, again with
the result that things don't cancel as expected.

-- glen