[Octave-bug-tracker] [bug #56936] format short displays 6, rather than 5, significant digits as documented

[Rik]

Follow-up Comment #1, bug #56936 (project octave):

Attached is a patch which resolves the issue for the example in this bug
report.  I only made the change to make_real_matrix_format because this is
still a work in progress.

What should happen when we have small numbers?  Should we end up printing a
lot of zeros?  That is what happens now.


x*1e-6
ans =

  -0.0000031416
              0
   0.0000031416


Before, we would switch over to scientific notation at a certain point.  These
are the results from version 3.4.3


octave-3.4.3:5> x*1e-3
ans =

  -0.0031416
   0.0000000
   0.0031416

octave-3.4.3:6> x*1e-4
ans =

  -3.1416e-04
   0.0000e+00
   3.1416e-04


I've asked on the mailing list for opinions.


(file #47565)
    _______________________________________________________

Additional Item Attachment:

File name: 56936.patch                    Size:1 KB
    <https://savannah.gnu.org/file/56936.patch?file_id=47565>



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?56936>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/