Re: munge-texi replacement

[John W. Eaton]

On 19-Apr-2012, Rik wrote:

| I already have a Perl replacement for munge-texi if we want to go that
| direction.
| 
| Advantages:
|   -- C++ is not the place for text manipulation
|   -- No need for compile rule in every *.txi -> *.texi transform
|   -- 1/4 lines of code to maintain, 2X faster
|       [maybe even faster used tied hash]
| 
| Disadvantages:
|   -- Requires Perl on build system
| 
| On the other hand, we already require Perl for building anyways so perhaps
| this isn't a big issue.

OK, I remember now that you proposed this change earlier and I opposed
it because of the Perl requirement.  But I suppose it would be OK and
since it is related to rebuilding documentation, it should not be
needed by someone building from a tarball.  So go ahead and check in
your change.

| > Are there other compiled programs that are used as part of the build
| > process?
| >
| gendoc in src/ and gethelp in scripts/ are both helper programs compiled
| only for the build.

The gethelp program should be easy to replace with an AWK or Perl
script.

The gendoc program is more complicated.  It is in C++ so that it can
take the contents of the doc strings in .df files that are extracted
from the C++ sources and perform backslash escape processing.  That
could be done by an AWK or Perl script, but then I'm not sure why we
have several scripts to perform this job instead of just one.

Currently we do the following:

  * Doc strings for DEFUNs are in the form

      DEFUN (all, args, ,
        "-*- texinfo -*-\n\
      @deftypefn  {Built-in Function} {} all (@var{x})\n\
      @deftypefnx {Built-in Function} {} all (@var{x}, @var{dim})\n\
      For a vector argument, return true (logical 1) if all elements of the 
vector\n\
      ...
      @seealso{any}\n\
      @end deftypefn")
      {
        ...
      }

   * The mkdefs script is used to extract the raw docstring from these
     files and generate .df files that have contents like this:

      // DO NOT EDIT!  Generated automatically by mkdefs.
       XDEFUN_FILE_NAME ("src/data.cc")
       XDEFUN_INTERNAL (all, args, , "-*- texinfo address@hidden  {Built-in 
Function} {} all (@var{x})address@hidden {Built-in Function} {} all (@var{x}, 
@var{dim})\nFor a vector argument, return true (logical 1) if all elements of 
the address@hidden@end deftypefn") 

   * The mkgendoc script concatenates the .df files together along
     with some extra info to create gendoc.cc.  That program is then
     executed to generate the DOCSTRINGS file.

   * If we are building Octave to use .oct files (the usual case) then
     a PKG_ADD file for the functions in the DLD-FUNCTIONS directory
     is also generated from all the .df files in the DLD-FUNCTIONS
     directory using the mk-pkg-add script.

OK, now I remember why this is broken up into so many steps.  If a
single source file changes, we only need to process that one file to
generate the corresponding .df file, and if the change in the source
file doesn't actually change the corresponding .df file, we can skip
building the documentation.  So I think we should continue to use this
multi-step process, but change mkgendoc+gendoc to a single script that
transforms the .df files directly to the DOCSTRINGS file.

jwe