[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: seq problem representing zero
From: |
Eric Blake |
Subject: |
Re: seq problem representing zero |
Date: |
Sat, 08 Oct 2005 09:32:51 -0600 |
User-agent: |
Mozilla Thunderbird 1.0.2 (Windows/20050317) |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
According to Simone Pascarosa on 10/8/2005 4:23 AM:
> For instance:
>> seq -3 0.6 1
> outputs:
> -3
> -2.4
> -1.8
> -1.2
> -0.6
> -1.11022e-16 <==
> 0.6
>
...
> Maybe this is just a problem with my math libraries or similar.
No, it is just a general inherent limitation of floating point
representation, which every computer programmer should be aware of. The
info pages also document this:
| Be careful when using `seq' with a fractional INCREMENT, otherwise
| you may see surprising results. Most people would expect to see `0.3'
| printed as the last number in this example:
|
| $ seq -s ' ' 0 .1 .3
| 0 0.1 0.2
|
| But that doesn't happen on most systems because `seq' is implemented
| using binary floating point arithmetic (via the C `double' type)--which
| means some decimal numbers like `.1' cannot be represented exactly.
| That in turn means some nonintuitive conditions like `.1 * 3 > .3' will
| end up being true.
>
> The problem disappears using --format option; for instance:
Correct, this is one way of forcing the computer to round to less
precision before outputting, and thus cover the inaccuracies inherent in
the full precision.
- --
Life is short - so eat dessert first!
Eric Blake address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDR+aj84KuGfSFAYARApOpAJ0UYRH2urPhTC71hJj0Y5LWQh3qOgCbBMYs
JLBmwx0ffsqIp1eoMgHIXnk=
=7IBF
-----END PGP SIGNATURE-----