Re: Test failure for num2str.m

[Daniel J Sebald]

On 09/06/2012 03:15 PM, Daniel J Sebald wrote:
> There is this test failure:
>
> > > > > > processing
> > > > > > /usr/local/src/octave/octave/octave/scripts/general/num2str.m
> ***** xtest
> assert (num2str (1e23), "100000000000000000000000");
> !!!!! known failure
> assert (num2str (1e23),"100000000000000000000000") expected
> 100000000000000000000000
> but got
> 99999999999999991611392
>
> We know the issue, i.e., limitation of floating point numbers.
>
> The question is, is this expected behavior? If so, I suggest altering
> the test so that it doesn't fail would be good. (Otherwise, the only fix
> is for the numeric class to store the string if it exceeds
> precision...which would be a ton of work.)
>
> But what of this FIXME?
>
> ## FIXME: Integers greater than bitmax() should be masked to show just
> ## 16 digits of precision.
> %!xtest
> %! assert (num2str (1e23), "100000000000000000000000");
>
> What is the point there?


I'm going to toss out some random results here with thoughts:

octave:753> num2str(1e22)
ans = 10000000000000000000000
octave:754> num2str(1e22-1)
ans = 10000000000000000000000

It turns out that 1e22, 1e21, etc represents the same number when 
truncated as floating point, but generally that isn't true for > 
bitmax() I take it.


octave:1> int64(1e23)
ans = 9223372036854775807
octave:2> num2str(int64(1e23))
ans = 9223372036854775808

It's peculiar that num2str isn't producing 9223372036854775807 in the 
second case.


octave:779> x = int64(1e23)
x = 9223372036854775807
octave:780> num2str(x)
ans = 9223372036854775808
octave:781> x = x - 1
x = 9223372036854775806
octave:782> num2str(x)
ans = 9223372036854775808
octave:783> x = x - 1
x = 9223372036854775805
octave:784> num2str(x)
ans = 9223372036854775808

Octave does handle 64-bit, but num2str doesn't handle 64-bit.


What I'm taking away from this is that num2str() could use a little work to

1) Limit the integer precision when displaying floating point numbers 
for which abs(x) > bitmax().
2) Update so that all integer precision is displayed when the input is 
of class intXX().

Perhaps this should go in the projects list under "Other Data Types".

Dan