[Octave-bug-tracker] [bug #39636] nbinpdf has floating point error

[anonymous]

Follow-up Comment #4, bug #39636 (project octave):

I understand about floating point issues and whether something is an integer
or not, but if I want to nbinpdf for real arguments, then I would expect
continuity.

octave:2> nbinpdf(0, 3, .5)
ans =  0.12500
octave:3> nbinpdf(0, 2.9999999999999, .5)
ans =  0.12495
octave:4> nbinpdf(0, 1/(1-2/3), .5)
ans =  0.18110           # discontinuous by bad floating pt luck
octave:5> nbinpdf(0, 3.0000000000001, .5)
ans =  0.12475


By definition in terms of the gamma function, 

nbinpdf(x,n,p) = gamma(x+n) / (gamma(x+1)*gamma(n)) * p^n*(1-p)^x

octave:10> x=0; n=1/(1-2/3); p=.5; gamma(x+n) / (gamma(x+1)*gamma(n)) *
p^n*(1-p)^x
ans =  0.12500

octave:11> x=0; n=1/(1-2/3); p=.5; gamma(x+n) / (gamma(x+1)*gamma(n)) *
p^n*(1-p)^x
ans =  0.12500   # gives the right answer



    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?39636>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/