Re: [bug-gawk] Gawk feature request - file list

[Andrew J. Schorr]

Hi Daniele,

On Wed, Feb 01, 2017 at 01:01:20PM +0100, Daniele Zanoli wrote:
> I'm an enthusiast user of gawk. I hope this is the best address to
> place a feature request (I could not find an "official" place for
> requests)

Thanks for your interest. I think this is the best (only?) address to use.  In
some cases, it may be useful to discuss issues in the comp.lang.awk usenet
group.

> Often I have to use gawk on a large number of files with a single
> execution of gawk (I have to compare some file contents) but it is
> very easy to reach the maximum length of the shell command line,
> since files are in different paths
> 
> So I think it would be useful (not only for me) a new command line
> switch to specify a file that contains the names of the files to
> process (e.g.
> gawk .. -list myfilelist.txt ...)

What platform/OS are you using? On my Linux CentOS 7.3 system, the
maximum shell command line length is theoretically over 2 million characters:

bash-4.2$ getconf ARG_MAX
2097152

In practice, I can't seem to get quite that much:

bash-4.2$ /bin/echo `gawk 'BEGIN {for (i = 0; i < 750000; i++) printf "1 "}'` | 
wc
      1  750000 1500000

So that's 1.5 million bytes. When I try 2 million, it craps out:

bash-4.2$ /bin/echo `gawk 'BEGIN {for (i = 0; i < 1000000; i++) printf "1 "}'`
Killed

I'm not sure why it was killed. For higher numbers, we run into the ARG_MAX 
limit:

bash-4.2$ /bin/echo `gawk 'BEGIN {for (i = 0; i < 1048576; i++) printf "1 "}'`
bash: /bin/echo: Argument list too long

Still, 1.5 million bytes is not too shabby...  How long an argument list are
you contemplating?

Note that the bash built-in echo command does not suffer from this limit:

bash-4.2$ echo `gawk 'BEGIN {for (i = 0; i < 5000000; i++) printf "1 "}'` | wc
      1 5000000 10000000

So it's important to use /bin/echo for this test.

Regards,
Andy