Re: implementing tw

[James Youngman]

On Thu, May 6, 2010 at 5:19 PM, Eric Blake <address@hidden> wrote:
> On today's Austin Group meeting (the people that standardize POSIX), it
> was mentioned that the next version of POSIX might do well to
> standardize tw(1).  http://www2.research.att.com/~gsf/man/man1/tw.html
> It seems like this would be a nice thing to add into findutils, as it
> combines the best of find(1) and xargs(1) into a single utility, whether
> or not POSIX ends up standardizing it.

Thanks for pointing this out.

I agree that it looks interesting.   There are a number of things I
would like to change or qualify about tw before standardising it.

1. We should leave room for extension of fields like type.
2. Timestamps are not integers any more, and we shouldn't assume they
are, even 64-bit integers.  ACL/attribute support seems missing too.
3. "local" as just an integer is probably a little simple for some
applications.  Perhaps we can leave room for generalising it.   The
special-casing of "local" when "local.foo" is defined is a bit ugly.
Perhaps everything should be "local.foo".
4. POSIX has no expr(3).   Do we need another language?   If we must
implement one, can we make it a useful one?
5. md5sum is known to be in danger of being too weak for some
purposes.  Can we select a scheme instead which is more future-proof,
for example "digest.somename" and allow digest.md5sum as a specific
instance?   Clearly this will mean other changes, for example
modifications to the default snapshot format.
6. Clearly implementors may also want to implement additional
expressions.  We should arrange a way for expressions with similar
intents but different semantics not to collide.   For example, by
allowing some kind of namespace qualification.   (Such a feature may
also make it possible for some implementations to add plugin support).
7. Standardising the LOCATE02 format as "gnu" is a bit inconvenient if
we're going to ever change the format (and we must if we will ever
become as efficient as mlocate).
8. The current specification doesn't really deal with the need to
define interfaces for parallelisation.  These days, computers often
have many-core CPUs and some have many disks (internally or
externally).   Where we are defining things that will constrain future
implementations, parallelism should be the default approach.   This is
a complex issue, but to pin down some of the problems, we will
probably want to evaluate the select and action functions in separate
threads.  Somewhat thornier is the fact that updating "parent" may
need to pass data from one thread to another.   Explicitly disallowing
assumptions about visit ordering would be a start.

>
> Does anyone want to tackle the implementation of tw as a summer project?
>
> --
> Eric Blake   address@hidden    +1-801-349-2682
> Libvirt virtualization library http://libvirt.org
>
>