Re: Single/Double precision NA values

[John W. Eaton]

On 30-Jul-2008, David Bateman wrote:

| John W. Eaton wrote:
| > On 29-Jul-2008, David Bateman wrote:
| >
| > | My current patch translates the old NA values to the new internal 
| > | representation when reading files, though I can't remember if fread does 
| > | it as well. However the code writes the new NA values to the files.
| >
| > It doesn't.  I looked, but I don't see a convenient place to insert
| > the conversion.
| >
| > I've applied the patch anyway, as it seems useful to have both
| > NA ("single") and single (NA) work correctly.
| >
| > jwe
| >
| >   
| Isn't the way to get fread to convert the NA values to just have a 
| specialization of
| t
| template <class NDArray, class double>
| octave_value
| do_read (octave_stream& strm, octave_idx_type nr, octave_idx_type nc, 
| octave_idx_type block_size,
|      octave_idx_type skip, bool do_float_fmt_conv,
|      oct_mach_info::float_forma from_flt_fmt, octave_idx_type& count)
| 
| in oct-stream.cc that performs the conversion of the old NA value to the 
| new one? This just means copying the do_read template and adding a copy 
| of lines at the end for the conversion. Or am I missing something?

As that is a fairly long function, I was hoping to find a way to avoid
duplicating most of it to just add a couple of lines.  For example,
the template function has a parameter "do_float_fmt_conv" that does
not apply to the instantiations for integer data types, but without
it, we would have to specialize for float vs. integer types too.
Could we add a parameter for doing NA translation?  OK, how about the
attached patch?

Also, I was unsure of precisely when to check for the old NA value.
Thinking about it now, I suppose the check should happen to the value
once it has been converted to the native floating point
representation.

jwe

# HG changeset patch
# User John W. Eaton <address@hidden>
# Date 1217434083 14400
# Node ID 89dd4531b26a0c8654ce6e9d5ffddcbbb37921d0
# Parent  8ccd9b0bf6bcc747aa43ade83e2a863f562bef2c
fread: translate NA values

diff --git a/src/ChangeLog b/src/ChangeLog
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,11 @@
+2008-07-30  John W. Eaton  <address@hidden>
+
+       * oct-stream.cc (do_read): New arg, do_NA_conv.
+       Perform NA translation if do_NA_conv is true.
+       (DO_READ_VAL_TEMPLATE, read_fptr): Include the new arg for do_read
+       in decl.
+       (octave_stream::read): Pass do_NA_conv to do_read.
+
 2008-07-29  David Bateman  <address@hidden>
 
        * data.cc (FNA): Add tests for conversion of single to double NA
diff --git a/src/oct-stream.cc b/src/oct-stream.cc
--- a/src/oct-stream.cc
+++ b/src/oct-stream.cc
@@ -3026,7 +3026,7 @@
 template <class RET_T, class READ_T>
 octave_value
 do_read (octave_stream& strm, octave_idx_type nr, octave_idx_type nc, 
octave_idx_type block_size,
-        octave_idx_type skip, bool do_float_fmt_conv,
+        octave_idx_type skip, bool do_float_fmt_conv, bool do_NA_conv,
         oct_mach_info::float_format from_flt_fmt, octave_idx_type& count)
 {
   octave_value retval;
@@ -3139,6 +3139,9 @@
                      dat = nda.fortran_vec ();
                    }
 
+                 if (do_NA_conv && __lo_ieee_is_old_NA (tmp))
+                   tmp = __lo_ieee_replace_old_NA (tmp);
+
                  dat[count++] = tmp;
 
                  elts_read++;
@@ -3190,7 +3193,7 @@
 
 #define DO_READ_VAL_TEMPLATE(RET_T, READ_T) \
   template octave_value \
-  do_read<RET_T, READ_T> (octave_stream&, octave_idx_type, octave_idx_type, 
octave_idx_type, octave_idx_type, bool, \
+  do_read<RET_T, READ_T> (octave_stream&, octave_idx_type, octave_idx_type, 
octave_idx_type, octave_idx_type, bool, bool, \
                          oct_mach_info::float_format, octave_idx_type&)
 
 // FIXME -- should we only have float if it is a different
@@ -3224,7 +3227,7 @@
 INSTANTIATE_DO_READ (charNDArray);
 INSTANTIATE_DO_READ (boolNDArray);
 
-typedef octave_value (*read_fptr) (octave_stream&, octave_idx_type, 
octave_idx_type, octave_idx_type, octave_idx_type, bool,
+typedef octave_value (*read_fptr) (octave_stream&, octave_idx_type, 
octave_idx_type, octave_idx_type, octave_idx_type, bool, bool,
                                   oct_mach_info::float_format ffmt, 
octave_idx_type&);
 
 NO_INSTANTIATE_ARRAY_SORT (read_fptr);
@@ -3314,10 +3317,13 @@
                                         || input_type == 
oct_data_conv::dt_single)
                                        && ffmt != float_format ());
 
+             bool do_NA_conv = (output_type == oct_data_conv::dt_double);
+
              if (fcn)
                {
                  retval = (*fcn) (*this, nr, nc, block_size, skip,
-                                  do_float_fmt_conv, ffmt, char_count);
+                                  do_float_fmt_conv, do_NA_conv,
+                                  ffmt, char_count);
 
                  // FIXME -- kluge!