Syntax checks in perl (was: Re: maint.mk syntax check problems)

[Martin von Gagern]

On 15.09.2011 11:37, Jim Meyering wrote:
> I'm sure that a perl-based
> implementation would be far more efficient, and probably faster
> even if the perl implementation doesn't run its tests in parallel.
> 
> Perl is well suited to this task.
> I'm sure some will object to Perl's syntax, but not I.

Not that I object to Perl, but as we are discussing different languages,
I'd like to offr Python as an alternative to consider. Main benefits:
- threading and therefore parallel tests
- clean exception handling, allowing checks to fail in deeply nested
  code and still recover to proceed with the next check
- somewhat mere legible syntax, I believe

I'm not sure if these benefits warrant adding another scripting language
both to the set of tools maintainers are expected to have around and to
the set of languages to maintain within gnulib.

> With a good Perl-based harness, I'll certainly be glad to phase
> out (of projects I maintain) the make-based tests.

Me, too.

On 15.09.2011 11:14, Stefano Lattarini wrote:
> About an yaer ago I had proposed a similar move for automake's own
> maintainer checks; see this RFC patch:
> 
> <http://lists.gnu.org/archive/html/automake-patches/2010-07/msg00081.html>

A good start! My main concerns are that for one, the framework might not
be flexible enough. The approach is well suited for checks processing
one line at a time, but checks that operate on the file text as a whole,
or that even pass the file to some other tool (e.g. indent), are rather
difficult to express. It would be nice if such test types could at least
be added later on with reasonable overhead.

To add some flexibility for future extensions, I believe that it would
be good to use some OOP approach, i.e. have test classes. Instances of
each class could be configured using keywords, which I very much like
about that approach. Most current checks, both from that batch and
maint.mk, would probably be instances of some regexp-checking class. But
others could be added later on.

Perhaps the regular expressions could operate on the whole file by
default, although that makes obtaining the offending pieces of code a
bit harder. But line breaks don't really matter in C, so looking even
accross them would be the right thing to do for many checks.

Some checks might operate on a different set of files than others, e.g.
generated files instead of version-controlled ones. Current maint.mk
does that for sc_po_check as well as those checks passing in_files to
_sc_search_regexp: sc_copyright_check, sc_Wundef_boolean and
sc_vulnerable_makefile_CVE-2009-4029. So file name alone isn't enough.

In case some more complicated checks want to exit a single check from
somewhere inside nested code, I would like to wrap all check execution
in "eval { ... }" so that a "die" within that code can be recovered
from. Although I must confess that this would make more sense with a
one-check-at-a-time look as the outermost one, whereas the proposed perl
script does one-file-at-a-time, saving io but causing repeated check calls.

The proposed script apparently has no means of configuration so far. I
guess it would be great if the configuration file were a perl script
itself, so it could not only modify the configuration affecting existing
checks, but even add completely new checks specific to a given project.

One more thing: at least for me, the above link does obfuscate large
parts of perl code which it incorrectly considers to contain e-mail
addresses, replacing those portions with "address@hidden". So some other
archived version of that mail must be used, e.g.
http://article.gmane.org/gmane.comp.sysutils.automake.patches/4302

Glad things were set in motion,
 Martin

signature.asc
Description: OpenPGP digital signature