Re: Command line parsing of ls with genparse: inline code

[Michael Geng]

On Sat, Jun 14, 2008 at 06:17:30PM +0200, Jim Meyering wrote:
> address@hidden (Michael Geng) wrote:
> > when I posted a patch around Christmas which showed how genparse
> > could generate the parser code for the ping command of the inetutils
> > Alfred Szmidt replied that in that example there are 2 loops (see
> > http://lists.gnu.org/archive/html/bug-inetutils/2007-12/msg00007.html):
> >
> > 1. The switch statement which sets the arg_t struct with the
> >    results of getopt_long in ping-clp.c
> >
> > 2. A list of if-clauses which sets the already existing variables
> >    in ping.c from the arg_t struct
> >
> > I now released a new version of genparse (0.7.6) which allows to put
> > the generated parser code inline of existing code, replacing almost
> > exactly the existing C code. This version has 2 extensions:
> >
> > 1. A __CODE__(statements) directive which allows to let the user define
> >    the code which is executed in the switch statement which evaluates
> >    the results of getopt_long.
> >
> > 2. A global directive #no_struct. If specified in the genparse file
> >    then no arg_t struct struct is generated.
> >
> > I uploaded a patch which shows how this works for the ls command of
> > the coreutils. It transforms ls.c.in which has the genparse file inline
> > into ls.c which has almost exactly the same command line parser code
> > inline as the present version of ls.c. The patch is targeted to today's
> > git version of the coreutils. I verified that it passes make distcheck.
> >
> > Please use the following links:
> >
> > patch:   
> > http://genparse.sourceforge.net/coreutils/genparse-ls-inline-git-01-19-2008.patch
> > ls.c.in: http://genparse.sourceforge.net/coreutils/ls.c.in (included in the 
> > patch)
> > ls.c     http://genparse.sourceforge.net/coreutils/ls.c    (included in the 
> > patch)
> 
> Hi Michael,
> 
> If you're still interested (and willing to work! ;-)
> I think we're getting close.  I hope you understand the delay.
> We're contemplating a massive change, and I want to be sure
> I don't end up regretting it.
> 
> Now that coreutils-6.12 is out, once two remaining points
> are addressed, then I think we'll finally be ready to switch
> a few trial programs.
> 
> Please add #line directives in the emitted C code, so that
> debuggers know the source file/line-numbers.
> 
> I'd really like to avoid having to move the primary source
> for 100+ C files from .c to .c.in files (or .in.c).
> I'm not sure how best to address that.  Here are some thoughts:
> 
> We might arrange for automake to emit an augmented .c-to-.o rule
> that invokes genparse along the way (this is where #line directives
> become essential -- the postprocessed intermediate .c file may not even
> exist after it's compiled.  Note however that that would add genparse
> as a *build-time* requirement, so isn't really an option.  The usual
> approach to solving this problem is to distribute both primary sources
> and generated sources (e.g., yacc and lex sources ship with generated
> .c files), but that would bloat the distribution tarballs.
> 
> One possibility would be to emit into a .h file the static
> definitions for functions like Cmdline.  Then there would be
> far less duplication.
> 
> Jim

Hi Jim,

I think emitting into a .h file would be a good solution. But wouldn't 
it then be better to directly add a genparse file (e.g. ls.gp) instead of 
extracting it from ls.c? ls.c would no longer have to be modified then. 
How about simply renaming the genparse generated ls-clp.c into a header
file, modifying it the same way the Makefile with my last patch already 
does and including it in ls.c? Here's a drawing of this idea:

       genparse             sed
ls.gp ----------> ls-clp.c -----> ls-genparse.h

This would mean that ls.c becomes more tiny and readable (because the
command line part is removed) but ls.gp will be added and in order not to
have genparse as a build-time requirement ls-genparse.h will also have to 
be added to the distribution. ls-clp.c becomes an intermediate file which 
need not be distributed, I hope that ls-clp.h will not be needed at all. 
There will be no code duplication except between the genparse file and the 
generated command line parser.

I haven't yet had time to test my idea, I hope I will find time to do that
in the next days.

Michael