Re: cygwin xargs limitation: ARG_MAX depends on command

[Bob Proulx]

Eric Blake wrote:
> Chris Faylor (one of the cygwin maintainers) proposed a benchmark that
> times the entire operation, not just the xargs invocation:
> http://sources.redhat.com/ml/cygwin-patches/2005-q3/msg00107.html
> 
> His numbers, with a modified xargs that ignored the _SC_ARG_MAX limit,
> showed improvement for cygwin processes (which can cope with the larger
> limits) as -s increased beyond 32k.

Interesting.  I don't have a Cygwin system to test this upon.  On my
system it has no effect that I can discern.  Timing the entire
pipeline or just the xargs with either small sized args or a file of
larger args such as a list of files on the system both give about the
same benchmark timing results.  (I understand that there is a large
fork penalty on ms systems and reducing them there as a much as
possible is desirable.)

> Forking on cygwin is very expensive.  I guess what I will end up doing is
> maintaining a cygwin-local patch that defaults to 32k+envsize (instead of
> the standard 128k+envsize) so that the default of not using -s always
> works on non-cygwin programs which have the 32k limit, but letting -s be
> ruled by sysconf(_SC_ARG_MAX), which Chris Faylor wants to leave at 1 meg

To be clear, I was not arguing against any of these proprosals.  I was
suggesting that if there was no performance benefit then doing the
simpler thing would be best.  It is almost always good to go the
simpler route.

> (or even increase, because he has instead proposed a fix for the upcoming
> cygwin 1.5.19 where ALL cygwin processes use the under-the-hood argument
> passing mechanism that allow them to exceed the Windows limitation,
> limited only by malloc, rather than the current 1.5.18 behavior of just
> the cygexec mount points getting the increased limit).  Yes, it means that
> xargs will get E2BIG if someone uses -s on a non-cygwin program, even
> though the POSIX rules are in place to try to prevent that, but it fits
> with cygwin's philosophy of favoring Unix emulation over native Windows
> apps, and at least the default will work everywhere.

Thanks for passing that information along.

Bob