[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #39636] nbinpdf has floating point error
From: |
anonymous |
Subject: |
[Octave-bug-tracker] [bug #39636] nbinpdf has floating point error |
Date: |
Wed, 31 Jul 2013 19:10:56 +0000 |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0 |
URL:
<http://savannah.gnu.org/bugs/?39636>
Summary: nbinpdf has floating point error
Project: GNU Octave
Submitted by: None
Submitted on: Wed 31 Jul 2013 07:10:55 PM UTC
Category: Octave Function
Severity: 3 - Normal
Priority: 5 - Normal
Item Group: Incorrect Result
Status: None
Assigned to: None
Originator Name: ken massey
Originator Email: address@hidden
Open/Closed: Open
Discussion Lock: Any
Release: 3.6.1
Operating System: GNU/Linux
_______________________________________________________
Details:
> nbinpdf(0,3,.5)
ans = 0.12500 # this is correct
> junk=1/(1-2/3) # equals 3 in exact arithmetic
junk = 3.0000
> nbinpdf(0,junk,.5)
ans = 0.18110 # this is incorrect
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?39636>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
- [Octave-bug-tracker] [bug #39636] nbinpdf has floating point error,
anonymous <=