PSPP-BUG: [bug #17241] Performance optimizations for procedures

[Ben Pfaff]

URL:
  <http://savannah.gnu.org/bugs/?func=detailitem&item_id=17241>

                 Summary: Performance optimizations for procedures
                 Project: PSPP
            Submitted by: blp
            Submitted on: Sunday 07/30/06 at 19:21
                Category: Other
                Severity: 1 - Wishlist
                  Status: None
             Assigned to: blp
             Open/Closed: Open
                 Release: None
                  Effort: 0.00

    _______________________________________________________

Details:

These are the realistic part of a more ambitious idea for performance
optimizations that I posted earlier to the pspp-dev mailing list.

1. Eliminate the need to cache procedure output.  Currently,
   every procedure reads the entire active file and writes out a
   new copy of it.  For big data sets that's often a huge waste
   of time and space.

   This was almost impossible to get right before I refactored
   the code for implementing procedures.  Now, it should only be
   a few days of work.

2. Implement an optimizer for transformations.  SELECT IF can
   often be moved earlier without changing any semantics.
   Creation of variables whose values are never used can be
   dropped entirely.  I suspect that this is especially valuable
   after #1 is implemented because #1 will retain any
   non-temporary transformations from one procedure to the next
   for re-execution, whereas currently that does not happen.

   Doing a basic job of this should be a few days of work.  It's
   probably not worth putting more into it than that (and maybe
   not that much) unless it is demonstrably useful.

2a. Implement caching of data between procedures.







    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?func=detailitem&item_id=17241>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/