[gnuastro-devel] [task #14779] Enable usage in HTCondor at configure time

[Mohammad Akhlaghi]

URL:
  <http://savannah.gnu.org/task/?14779>

                 Summary: Enable usage in HTCondor at configure time
                 Project: GNU Astronomy Utilities
            Submitted by: makhlaghi
            Submitted on: Thu 21 Dec 2017 12:08:19 AM CET
         Should Start On: Wed 20 Dec 2017 12:00:00 AM CET
   Should be Finished on: Wed 20 Dec 2017 12:00:00 AM CET
                Category: Installation
                Priority: 5 - Normal
              Item Group: Enhancement
                  Status: Postponed
                 Privacy: Public
        Percent Complete: 0%
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
                  Effort: 0.00

    _______________________________________________________

Details:

>From its webpage <https://research.cs.wisc.edu/htcondor/description.html>:
"HTCondor is a specialized workload management system for compute-intensive
jobs. Like other full-featured batch systems, HTCondor provides a job queueing
mechanism, scheduling policy, priority scheme, resource monitoring, and
resource management. Users submit their serial or parallel jobs to HTCondor,
HTCondor places them into a queue, chooses when and where to run the jobs
based upon a policy, carefully monitors their progress, and ultimately informs
the user upon completion."

In its standard universe
<http://research.cs.wisc.edu/htcondor/manual/v8.7/2_4Running_Job.html#SECTION00341100000000000000>,
HTCondor will create checkpoints, to migrate a (partially complete) job from
one system to another when necessary and thus not needing to redo the job. It
also allows "remote system calls". The latter is great in efficiency, because
in practice it will trick a program on one machine to use the resources of
another system without knowing it (very good for processing efficiency). 

However, to benefit from these features of this "standard universe", a program
should be compiled with the `condor_compile' command to re-link it with
HTCondor libraries (see the description
<http://research.cs.wisc.edu/htcondor/manual/v8.7/2_4Running_Job.html#SECTION00341100000000000000>
for examples). Some special features (all of which, if present, are optional
in Gnuastro)) like `mmap' are not allowed and libraries must be statically
linked.

All these HTCondor requirements to allow usage in a "standard universe" can
easily be implemented in Gnuastro with a configure time option. For example
`--enable-htcondor'. When configured with this option, the compiled Gnuastro
executables can benefit from these great features.

This can include BuildProgram, so even programs that the user defines
themselves using the Gnuastro Library can be run with the features of the
"Standard universe".




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/task/?14779>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/