Re: cygwin xargs limitation: ARG_MAX depends on command

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Bob Proulx on 9/5/2005 3:33 PM:
> If it is not too much trouble could you do post some simple benchmarks
> showing that larger buffer sizes are significantly better than the 32k
> buffer sizes?  If so then it would help to promote your case.

Chris Faylor (one of the cygwin maintainers) proposed a benchmark that
times the entire operation, not just the xargs invocation:
http://sources.redhat.com/ml/cygwin-patches/2005-q3/msg00107.html

His numbers, with a modified xargs that ignored the _SC_ARG_MAX limit,
showed improvement for cygwin processes (which can cope with the larger
limits) as -s increased beyond 32k.

> 
> It just very quickly does not matter.  Let me guess, without data and
> perhaps wrongly, that this will not be a big performance difference on
> cygwin either.  So probably on cygwin I would go ahead and make that
> size restriction so that it works correctly all of the time.  But I am
> not mothering over that platform and I know that those that do will
> want to do better so personally I sympathize.  But it just does not
> seem to me like a performance lever.

Forking on cygwin is very expensive.  I guess what I will end up doing is
maintaining a cygwin-local patch that defaults to 32k+envsize (instead of
the standard 128k+envsize) so that the default of not using -s always
works on non-cygwin programs which have the 32k limit, but letting -s be
ruled by sysconf(_SC_ARG_MAX), which Chris Faylor wants to leave at 1 meg
(or even increase, because he has instead proposed a fix for the upcoming
cygwin 1.5.19 where ALL cygwin processes use the under-the-hood argument
passing mechanism that allow them to exceed the Windows limitation,
limited only by malloc, rather than the current 1.5.18 behavior of just
the cygexec mount points getting the increased limit).  Yes, it means that
xargs will get E2BIG if someone uses -s on a non-cygwin program, even
though the POSIX rules are in place to try to prevent that, but it fits
with cygwin's philosophy of favoring Unix emulation over native Windows
apps, and at least the default will work everywhere.

- --
Life is short - so eat dessert first!

Eric Blake             address@hidden
volunteer cygwin package maintainer for findutils
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDMVyb84KuGfSFAYARAsbtAJ42HNzQ1eQjU41ohztVbyE2B+hsvgCdFBBd
W8o+68ICWU+fzA1y+KSXK1I=
=5rnV
-----END PGP SIGNATURE-----