[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results
From: |
Kai Torben Ohlhus |
Subject: |
[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results |
Date: |
Fri, 24 Aug 2018 18:54:15 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36 |
Follow-up Comment #2, bug #54567 (project octave):
Matlab R2018a returns "140" in both cases of the original comment.
The patch file #44857 of comment #1 works
- retval = sum (nth_element (x, k:k+1, dim), dim, "native") / 2;
+ retval = mean (nth_element (x, k:k+1, dim), dim, "native");
But I think it makes things more complicated and maybe slower for large data?!
Basically, mean() would perform the computation with double precision and
finally convert the result to the input data type. Why not doing the very
same without additional function calls in one line and tidy up the nasty
"logical" exception:
- retval = sum (nth_element (x, k:k+1, dim), dim, "native") / 2;
- if (islogical (x))
- retval = logical (retval);
- endif
+ ## Perform the averaging in double precision to avoid intermediate
+ ## overflows of finite data types like "int8".
+ retval = cast (sum (nth_element (x, k:k+1, dim), dim) / 2, class (x));
I thought about other solutions from the "real plain dump" integer world (like
C/C++) but these would not apply for Octave... This I am afraid to stay to
this solution.
What do you think about this Dan Sebald and Etienne de Foras?
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?54567>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, anonymous, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results,
Kai Torben Ohlhus <=
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Kai Torben Ohlhus, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Kai Torben Ohlhus, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Michael Godfrey, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Rik, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/26