[Octave-bug-tracker] [bug #47964] rats test failure on Debian i386

[Mike Miller]

URL:
  <http://savannah.gnu.org/bugs/?47964>

                 Summary: rats test failure on Debian i386
                 Project: GNU Octave
            Submitted by: mtmiller
            Submitted on: Thu 19 May 2016 10:58:41 AM PDT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: Need Info
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: dev
        Operating System: GNU/Linux

    _______________________________________________________

Details:

I see a new test failure (meaning the same test doesn't fail on the stable
branch) in pr-output.cc:


>> test pr-output.cc-tst
***** test
 v = 1 / double (intmax);
 err = v - str2num (rats(v, 12));
 assert (err, 0, 4 * eps);
!!!!! test failed
ASSERT errors for:  assert (err,0,4 * eps)

  Location  |  Observed  |  Expected  |  Reason
     ()       4.6566e-10       0         Abs err 4.6566e-10 exceeds tol
8.8818e-16


The underlying cause seems to be this difference between 4.0.2 and the default
branch (both on Debian unstable i386):


>> rats (1 / double (intmax), 12)
ans = 1/2147483647
>> rats (1 / double (intmax - 1), 12)
ans = 1/2147483646


vs


>> rats (1 / double (intmax), 12)
ans =            0
>> rats (1 / double (intmax - 1), 12)
ans = 1/2147483646


Any thoughts on where this difference came in on the default branch? I
remember some discussion about intmax being off by one, that's all I can think
of at the moment.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47964>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/