[Octave-bug-tracker] [bug #39332] quantiz function gives wrong results with scalar or single value table

[Chris van der Westhuizen]

URL:
  <http://savannah.gnu.org/bugs/?39332>

                 Summary: quantiz function gives wrong results with scalar or
single value table
                 Project: GNU Octave
            Submitted by: chrisvdw
            Submitted on: Mon 24 Jun 2013 04:19:16 PM GMT
                Category: Octave Forge Package
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: chrisvdw
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 3.6.4
        Operating System: Microsoft Windows

    _______________________________________________________

Details:

I am referring to the function in Communications package v. 1.1.1.

Actually I think I found two bugs at the same time, this was by trying to run
the DVB-T2 Common Simulation Platform (Sourceforge), where they used the
function like this:

rxData = ~(quantiz(rxData,0));

The intention is to get logical 0's and 1's.

Bug #1:

Try this

octave-3.6.4> quantiz([-Inf -Inf Inf Inf -Inf Inf -Inf Inf], 0)
ans =

   1
   1
   0
   0
   1
   0
   1
   0

The 1's and 0's are inverted.  For -Inf, refer to the first bullet point in
Mathworks online description. The answer should be a 0.  Note that in
partition(x), x refers to the Matlab 1-based indices of the table, not the
0-based return values.  
I think for n=1, the second bullet point never applies. For Inf, we should
look at the 3rd bullet point, where sig > partition(n).

I agree using the function with length 1 table is not a very rational thing to
do, but this is the code I found!  I am fully willing to help test it, if
possible.

Bug #2:

In the above the input is a row, the output should also be a row.  (If the
input is a column, the output should be a column.)





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?39332>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/