[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: saveing/loading symbol table of annymous functions
|
From: |
David Bateman |
|
Subject: |
Re: saveing/loading symbol table of annymous functions |
|
Date: |
Wed, 11 Apr 2007 23:40:55 +0200 |
|
User-agent: |
Thunderbird 1.5.0.7 (X11/20060921) |
John W. Eaton wrote:
> I'm not sure why we were duplicating the data for a function directly
> in the tree_anon_fcn_handle class, so I've checked in the following
> change. Now the tree_anon_fcn_handle object just contains an
> octave_user_function object that contains the parameter list, function
> body, return list, and symbol table.
>
> The octave_fcn_handle class has an octave_value object that contains
> the function. You can extract a pointer to the user function object
> and then get the symbol table from there using the new
> octave_user_function::sym_tab method. Does that give you everything
> you need to write the load/save functions for anonymous function
> handles?
>
> Thanks,
>
> jwe
>
>
Ok, I took a quick look at how to go about saving and then reloading the
local symbol table of a function handle, and have implemented it for the
case of "save {-text|-binary|-hdf5}" in the attached patch, including
unit test code. I treated the fact that for
a = 2;
f = @(x) = a+x;
f(1)
save -text test.mat f
the local symbol table of "f" after the evaluation "f(1)" has two
undefined symbol records "__retval__" and "x" by excluding undefined
symbol records when saving anonymous function handles.
I also tried to handle backwards compatibility by only saving the symbol
table if needed. Therefore
g = @(x) 2 * x;
save -text test.mat g
can be saved if this patch is applied and read by older versions of
octave. The reverse is also true. However, anonymous function handles
such as "f" above were never saved correct as the symbol table wasn't
saved. So there is no backward compatibility in that case. To do this
for the binary type I included the length of the symbol table in the
function name like "@<anonymous> 2" if the symbol table has two
elements. For text files I added a keyword "length" and the hdf5 files
added an attribute "SYMBOL_TABLE" with the length of the symbol table.
I look at getting function handles to be loaded and saved in matlab v5
files next..
Cheers
David
--
David Bateman address@hidden
Motorola Labs - Paris +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin +33 6 72 01 06 33 (Mob)
91193 Gif-Sur-Yvette FRANCE +33 1 69 35 77 01 (Fax)
The information contained in this communication has been classified as:
[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary
*** ./src/ov-fcn-handle.cc.orig8 2007-04-11 17:59:27.079732835 +0200
--- ./src/ov-fcn-handle.cc 2007-04-11 23:33:47.141287898 +0200
***************
*** 46,54 ****
--- 46,56 ----
#include "pt-assign.h"
#include "variables.h"
#include "parse.h"
+ #include "unwind-prot.h"
#include "byte-swap.h"
#include "ls-oct-ascii.h"
+ #include "ls-oct-binary.h"
#include "ls-hdf5.h"
#include "ls-utils.h"
***************
*** 144,150 ****
}
bool
! octave_fcn_handle::save_ascii (std::ostream& os, bool&)
{
os << nm << "\n";
--- 146,152 ----
}
bool
! octave_fcn_handle::save_ascii (std::ostream& os, bool& infnan_warned)
{
os << nm << "\n";
***************
*** 152,157 ****
--- 154,187 ----
{
print_raw (os, true);
os << "\n";
+
+ if (fcn.is_undefined())
+ return false;
+
+ octave_user_function *f = fcn.user_function_value ();
+
+ Array<symbol_record *> vars = f->sym_tab()->symbol_list();
+ octave_idx_type varlen = vars.length();
+
+ // Exclude undefined values like __retval__
+ for (int i = 0; i < vars.length(); i++)
+ {
+ if (! vars(i)->is_defined ())
+ varlen--;
+ }
+
+ if (varlen > 0)
+ {
+ os << "# length: " << varlen << "\n";
+
+ for (int i = 0; i < vars.length(); i++)
+ {
+ if (vars(i)->is_defined () &&
+ ! save_ascii_data (os, vars(i)->def(), vars(i)->name(),
+ infnan_warned, false, 0))
+ return os;
+ }
+ }
}
return true;
***************
*** 160,169 ****
--- 190,201 ----
bool
octave_fcn_handle::load_ascii (std::istream& is)
{
+ bool success = true;
is >> nm;
if (nm == "@<anonymous>")
{
+ octave_idx_type len = 0;
char c;
std::ostringstream buf;
***************
*** 187,206 ****
}
}
! int parse_status;
! octave_value anon_fcn_handle = eval_string (buf.str (), true,
! parse_status);
!
! if (parse_status == 0)
! {
! octave_fcn_handle *fh = anon_fcn_handle.fcn_handle_value ();
! if (fh)
! fcn = fh->fcn;
else
! return false;
}
else
! return false;
}
else
{
--- 219,301 ----
}
}
! std::streampos pos = is.tellg ();
! symbol_table *local_sym_tab = 0;
!
! if (extract_keyword (is, "length", len, true) && len >= 0)
! {
! if (len > 0)
! {
! octave_idx_type nlen = len;
! if (nlen % 2)
! nlen++;
!
! local_sym_tab = new symbol_table (nlen, "LOCAL");
!
! for (octave_idx_type i = 0; i < len; i++)
! {
! octave_value t2;
! bool dummy;
!
! std::string name
! = read_ascii_data (is, std::string (), dummy, t2, i);
!
! if (!is)
! {
! error ("load: failed to load anonymous function handle");
! break;
! }
!
! symbol_record *sr = local_sym_tab->lookup (name, true);
!
! if (sr)
! sr->define (t2);
! else
! {
! error ("load: failed to load anonymous function handle");
! success = false;
! break;
! }
! }
! }
! }
! else
! {
! is.seekg (pos);
! is.clear ();
! }
!
! if (is && success)
! {
! unwind_protect::begin_frame ("anon_ascii_load");
! unwind_protect_ptr (curr_sym_tab);
!
! if (local_sym_tab)
! curr_sym_tab = local_sym_tab;
!
! int parse_status;
! octave_value anon_fcn_handle =
! eval_string (buf.str (), true, parse_status);
!
! if (parse_status == 0)
! {
! octave_fcn_handle *fh =
! anon_fcn_handle.fcn_handle_value ();
! if (fh)
! fcn = fh->fcn;
! else
! success = false;
! }
else
! success = false;
!
! unwind_protect::run_frame ("anon_ascii_load");
}
else
! success = false;
!
! if (local_sym_tab)
! delete local_sym_tab;
}
else
{
***************
*** 208,241 ****
if (! fcn.is_function ())
{
error ("function handle points to non-existent function");
! return false;
}
}
! return true;
}
bool
! octave_fcn_handle::save_binary (std::ostream& os, bool&)
{
- int32_t tmp = nm.length ();
- os.write (reinterpret_cast<char *> (&tmp), 4);
- os.write (nm.c_str (), nm.length ());
if (nm == "@<anonymous>")
{
std::ostringstream buf;
print_raw (buf, true);
std::string stmp = buf.str ();
tmp = stmp.length ();
os.write (reinterpret_cast<char *> (&tmp), 4);
os.write (stmp.c_str (), stmp.length ());
}
return true;
}
bool
octave_fcn_handle::load_binary (std::istream& is, bool swap,
! oct_mach_info::float_format)
{
int32_t tmp;
if (! is.read (reinterpret_cast<char *> (&tmp), 4))
--- 303,399 ----
if (! fcn.is_function ())
{
error ("function handle points to non-existent function");
! success = false;
}
}
! return success;
}
+ /*
+
+ %!test
+ %! a = 2;
+ %! f = @(x) a + x;
+ %! g = @(x) 2 * x;
+ %! f2 = f;
+ %! g2 = g;
+ %! unwind_protect
+ %! save -text test.mat f2 g2
+ %! clear f2 g2
+ %! load test.mat
+ %! assert (f(2),f2(2));
+ %! assert (g(2),g2(2));
+ %! unlink ("test.mat");
+ %! save -text test.mat f2 g2
+ %! unwind_protect_cleanup
+ %! unlink ("test.mat");
+ %! end_unwind_protect
+
+ */
+
bool
! octave_fcn_handle::save_binary (std::ostream& os, bool& save_as_floats)
{
if (nm == "@<anonymous>")
{
+ std::ostringstream nmbuf;
+
+ if (fcn.is_undefined())
+ return false;
+
+ octave_user_function *f = fcn.user_function_value ();
+
+ Array<symbol_record *> vars = f->sym_tab()->symbol_list();
+ octave_idx_type varlen = vars.length();
+
+ // Exclude undefined values like __retval__
+ for (int i = 0; i < vars.length(); i++)
+ {
+ if (! vars(i)->is_defined ())
+ varlen--;
+ }
+
+ if (varlen > 0)
+ nmbuf << nm << " " << varlen;
+ else
+ nmbuf << nm;
+
+ std::string buf_str = nmbuf.str();
+ int32_t tmp = buf_str.length ();
+ os.write (reinterpret_cast<char *> (&tmp), 4);
+ os.write (buf_str.c_str (), buf_str.length ());
+
std::ostringstream buf;
print_raw (buf, true);
std::string stmp = buf.str ();
tmp = stmp.length ();
os.write (reinterpret_cast<char *> (&tmp), 4);
os.write (stmp.c_str (), stmp.length ());
+
+ if (varlen > 0)
+ {
+ for (int i = 0; i < vars.length(); i++)
+ {
+ if (vars(i)->is_defined () &&
+ ! save_binary_data (os, vars(i)->def(), vars(i)->name(),
+ "", 0, save_as_floats))
+ return os;
+ }
+ }
+ }
+ else
+ {
+ int32_t tmp = nm.length ();
+ os.write (reinterpret_cast<char *> (&tmp), 4);
+ os.write (nm.c_str (), nm.length ());
}
return true;
}
bool
octave_fcn_handle::load_binary (std::istream& is, bool swap,
! oct_mach_info::float_format fmt)
{
int32_t tmp;
if (! is.read (reinterpret_cast<char *> (&tmp), 4))
***************
*** 250,257 ****
if (! is)
return false;
! if (nm == "@<anonymous>")
{
if (! is.read (reinterpret_cast<char *> (&tmp), 4))
return false;
if (swap)
--- 408,424 ----
if (! is)
return false;
! if (nm.length() >= 12 && nm.substr (0, 12) == "@<anonymous>")
{
+ octave_idx_type len = 0;
+
+ if (nm.length() > 12)
+ {
+ std::istringstream nm_is (nm.substr(12));
+ nm_is >> len;
+ nm = nm.substr(0,12);
+ }
+
if (! is.read (reinterpret_cast<char *> (&tmp), 4))
return false;
if (swap)
***************
*** 260,278 ****
OCTAVE_LOCAL_BUFFER (char, ctmp2, tmp+1);
is.read (ctmp2, tmp);
! int parse_status;
! octave_value anon_fcn_handle = eval_string (ctmp2, true, parse_status);
! if (parse_status == 0)
{
! octave_fcn_handle *fh = anon_fcn_handle.fcn_handle_value ();
! if (fh)
! fcn = fh->fcn;
else
! return false;
}
! else
! return false;
}
else
{
--- 427,504 ----
OCTAVE_LOCAL_BUFFER (char, ctmp2, tmp+1);
is.read (ctmp2, tmp);
! symbol_table *local_sym_tab = 0;
! bool success = true;
! if (len > 0)
! {
! octave_idx_type nlen = len;
! if (nlen % 2)
! nlen++;
!
! local_sym_tab = new symbol_table (nlen, "LOCAL");
!
! for (octave_idx_type i = 0; i < len; i++)
! {
! octave_value t2;
! bool dummy;
! std::string doc;
!
! std::string name =
! read_binary_data (is, swap, fmt, std::string (),
! dummy, t2, doc);
!
! if (!is)
! {
! error ("load: failed to load anonymous function handle");
! break;
! }
!
! symbol_record *sr = local_sym_tab->lookup (name, true);
!
! if (sr)
! {
! sr->define (t2);
! sr->document (doc);
! }
! else
! {
! error ("load: failed to load anonymous function handle");
! success = false;
! break;
! }
! }
! }
! if (is && success)
{
! unwind_protect::begin_frame ("anon_binary_load");
! unwind_protect_ptr (curr_sym_tab);
!
! if (local_sym_tab)
! curr_sym_tab = local_sym_tab;
!
! int parse_status;
! octave_value anon_fcn_handle =
! eval_string (ctmp2, true, parse_status);
!
! if (parse_status == 0)
! {
! octave_fcn_handle *fh = anon_fcn_handle.fcn_handle_value ();
! if (fh)
! fcn = fh->fcn;
! else
! success = false;
! }
else
! success = false;
!
! unwind_protect::run_frame ("anon_binary_load");
}
!
! if (local_sym_tab)
! delete local_sym_tab;
!
! return success;
}
else
{
***************
*** 286,295 ****
return true;
}
#if defined (HAVE_HDF5)
bool
octave_fcn_handle::save_hdf5 (hid_t loc_id, const char *name,
! bool /* save_as_floats */)
{
hid_t group_hid = -1;
group_hid = H5Gcreate (loc_id, name, 0);
--- 512,543 ----
return true;
}
+ /*
+
+ %!test
+ %! a = 2;
+ %! f = @(x) a + x;
+ %! g = @(x) 2 * x;
+ %! f2 = f;
+ %! g2 = g;
+ %! unwind_protect
+ %! save -binary test.mat f2 g2
+ %! clear f2 g2
+ %! load test.mat
+ %! assert (f(2),f2(2));
+ %! assert (g(2),g2(2));
+ %! unlink ("test.mat");
+ %! save -binary test.mat f2 g2
+ %! unwind_protect_cleanup
+ %! unlink ("test.mat");
+ %! end_unwind_protect
+
+ */
+
#if defined (HAVE_HDF5)
bool
octave_fcn_handle::save_hdf5 (hid_t loc_id, const char *name,
! bool save_as_floats)
{
hid_t group_hid = -1;
group_hid = H5Gcreate (loc_id, name, 0);
***************
*** 355,360 ****
--- 603,661 ----
}
H5Dclose (data_hid);
+
+ octave_user_function *f = fcn.user_function_value ();
+ Array<symbol_record *> vars = f->sym_tab()->symbol_list();
+ octave_idx_type varlen = vars.length();
+
+ // Exclude undefined values like __retval__
+ for (int i = 0; i < vars.length(); i++)
+ {
+ if (! vars(i)->is_defined ())
+ varlen--;
+ }
+
+ if (varlen > 0)
+ {
+ hid_t as_id = H5Screate (H5S_SCALAR);
+
+ if (as_id >= 0)
+ {
+ hid_t a_id = H5Acreate (group_hid, "SYMBOL_TABLE",
+ H5T_NATIVE_IDX, as_id, H5P_DEFAULT);
+
+ if (a_id >= 0)
+ {
+ retval = (H5Awrite (a_id, H5T_NATIVE_IDX, &varlen) >= 0);
+
+ H5Aclose (a_id);
+ }
+ else
+ retval = false;
+
+ H5Sclose (as_id);
+ }
+ else
+ retval = false;
+
+ data_hid = H5Gcreate (group_hid, "symbol table", 0);
+ if (data_hid < 0)
+ {
+ H5Sclose (space_hid);
+ H5Tclose (type_hid);
+ H5Gclose (group_hid);
+ return false;
+ }
+
+ for (int i = 0; i < vars.length(); i++)
+ {
+ if (vars(i)->is_defined () &&
+ ! add_hdf5_data (data_hid, vars(i)->def(), vars(i)->name(),
+ "", false, save_as_floats))
+ break;
+ }
+ H5Gclose (data_hid);
+ }
}
H5Sclose (space_hid);
***************
*** 366,372 ****
bool
octave_fcn_handle::load_hdf5 (hid_t loc_id, const char *name,
! bool /* have_h5giterate_bug */)
{
hid_t group_hid, data_hid, space_hid, type_hid, type_class_hid, st_id;
hsize_t rank;
--- 667,673 ----
bool
octave_fcn_handle::load_hdf5 (hid_t loc_id, const char *name,
! bool have_h5giterate_bug)
{
hid_t group_hid, data_hid, space_hid, type_hid, type_class_hid, st_id;
hsize_t rank;
***************
*** 490,510 ****
return false;
}
H5Dclose (data_hid);
- H5Tclose (st_id);
! int parse_status;
! octave_value anon_fcn_handle = eval_string (fcn_tmp, true,
parse_status);
! if (parse_status == 0)
{
! octave_fcn_handle *fh = anon_fcn_handle.fcn_handle_value ();
! if (fh)
! fcn = fh->fcn;
else
! return false;
}
! else
! return false;
}
else
{
--- 791,911 ----
return false;
}
H5Dclose (data_hid);
! symbol_table *local_sym_tab = 0;
! bool success = true;
! octave_idx_type len = 0;
!
! // we have to pull some shenanigans here to make sure
! // HDF5 doesn't print out all sorts of error messages if we
! // call H5Aopen for a non-existing attribute
!
! H5E_auto_t err_func;
! void *err_func_data;
!
! // turn off error reporting temporarily, but save the error
! // reporting function:
! H5Eget_auto (&err_func, &err_func_data);
! H5Eset_auto (0, 0);
! hid_t attr_id = H5Aopen_name (group_hid, "SYMBOL_TABLE");
!
! if (attr_id >= 0)
! {
! if (H5Aread (attr_id, H5T_NATIVE_IDX, &len) < 0)
! success = false;
!
! H5Aclose (attr_id);
! }
!
! // restore error reporting:
! H5Eset_auto (err_func, err_func_data);
!
! if (len > 0 && success)
! {
! octave_idx_type nlen = len;
! if (nlen % 2)
! nlen++;
!
! local_sym_tab = new symbol_table (nlen, "LOCAL");
!
! #ifdef HAVE_H5GGET_NUM_OBJS
! hsize_t num_obj = 0;
! data_hid = H5Gopen (group_hid, "symbol table");
! H5Gget_num_objs (data_hid, &num_obj);
! H5Gclose (data_hid);
!
! if (num_obj != static_cast<hsize_t>(len))
! {
! error ("load: failed to load anonymous function handle");
! success = false;
! }
! #endif
!
! if (! error_state)
! {
! hdf5_callback_data dsub;
! int current_item = 0;
! for (octave_idx_type i = 0; i < len; i++)
! {
! if (H5Giterate (group_hid, "symbol table", ¤t_item,
! hdf5_read_next_data, &dsub) <= 0)
! {
! error ("load: failed to load anonymous function handle");
! success = false;
! break;
! }
!
! if (have_h5giterate_bug)
! current_item++; // H5Giterate returns last index processed
!
! symbol_record *sr = local_sym_tab->lookup (dsub.name, true);
!
! if (sr)
! sr->define (dsub.tc);
! else
! {
! error ("load: failed to load anonymous function handle");
! success = false;
! break;
! }
! }
! }
! }
!
! H5Tclose (st_id);
! H5Gclose (group_hid);
!
! if (success)
{
! unwind_protect::begin_frame ("anon_hdf5_load");
! unwind_protect_ptr (curr_sym_tab);
!
! if (local_sym_tab)
! curr_sym_tab = local_sym_tab;
!
! int parse_status;
! octave_value anon_fcn_handle =
! eval_string (fcn_tmp, true, parse_status);
!
! if (parse_status == 0)
! {
! octave_fcn_handle *fh = anon_fcn_handle.fcn_handle_value ();
! if (fh)
! fcn = fh->fcn;
! else
! success = false;
! }
else
! success = false;
!
! unwind_protect::run_frame ("anon_hdf5_load");
}
!
! if (local_sym_tab)
! delete local_sym_tab;
!
! return success;
}
else
{
***************
*** 518,523 ****
--- 919,948 ----
return true;
}
+
+ /*
+
+ %!test
+ %! if (!isempty(findstr(octave_config_info ("DEFS"),"HAVE_HDF5")))
+ %! a = 2;
+ %! f = @(x) a + x;
+ %! g = @(x) 2 * x;
+ %! f2 = f;
+ %! g2 = g;
+ %! unwind_protect
+ %! save -hdf5 test.mat f2 g2
+ %! clear f2 g2
+ %! load test.mat
+ %! assert (f(2),f2(2));
+ %! assert (g(2),g2(2));
+ %! unlink ("test.mat");
+ %! save -hdf5 test.mat f2 g2
+ %! unwind_protect_cleanup
+ %! unlink ("test.mat");
+ %! end_unwind_protect
+ %! endif
+
+ */
#endif
void
2007-04-11 David Bateman <address@hidden>
* ov-fcn-handle.cc (octave_fcn_handle_save_ascii,
octave_fcn_handle::load_ascii, octave_fcn_handle::save_binary,
octave_fcn_handle::load_binary, octave_fcn_handle::save_hdf5,
octave_fcn_handle::load_hdf5): Save and reload the local symbol
table of the user function associated with anonymous function
handles. Attempt to maintain backward and forward compatibility.