[Octave-bug-tracker] [bug #33104] mat2str returns ugly result (different to Matlab)

[Jordi Gutiérrez Hermoso]

Follow-up Comment #10, bug #33104 (project octave):

Ben Abbot wrote:
> My thought was that the number of digits displayed should not exceed
> the numeric precision. In general, we can only be confident in the
> first 15 digits, correct?

As I said, it really is a matter of preference, but as far as defaults
go, since the 16th digit has a good chance of being accurate, as 2^52
is approximately 4.5e15, my own preference is to give as much accuracy
as possible. The 17th digit of course can't be accurate except by
fluke, but the 16th digit has about a 45% chance of being accurate. My
own preference is that by default that extra digit be given and that
the user should understand the caveats that the 16th digit could be
wrong about half the time.

Hiding that digit so that users don't complain about 0.7 not being
exactly representable in IEEE 754 arithmetic seems a little
patronising to me. I'd rather explain a million times how hardware
floats work than to alleviate user complaints about them.

Then again, looking at other Octave functions, I see that format long
and save apparently prefers to round to 15 digits, or rather, prefers
to count the leading zero as part of the digits it displays. Perhaps
with consistency for those functions, this patch is correct, and I'll
stop being contrarian.


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?33104>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/