Re: GNU Parallel Bug Reports unexpected failure due to ssh line length exceeded error

[E V]

It would be nice if gnu parallel automatically put as many arguments
as it could, size wise, up to a maximum of the -N setting. My actual
use case is a file with significantly longer, variable length path
names in it and it's frustrating to have one run work with, say, -N
700, then the next get stuck and make me go find a working -N again.

On Thu, Jan 30, 2014 at 6:23 PM, Ole Tange <address@hidden> wrote:
> The problem is that if the maximal command line length is 128 KB
> (which it is on a normal GNU/Linux installation), you can only fit a
> bit over 7000 of those numbers on the command line.
>
> But GNU Parallel ought to give a reasonable error message instead of
> just spiraling into a loop of death.
>
> Work around: -N7000
>
>
> /Ole
>
> On Tue, Jan 28, 2014 at 9:15 PM, E V <address@hidden> wrote:
>> $ parallel --version
>> GNU parallel 20140122
>>
>> $ seq 1000000000000000 1000000001000000 > lots_o_number.txt
>>
>> $ parallel --sshloginfile ~/sshparallellogins -N 10000 echo {} <
>> lots_o_number.txt > lots_o.log
>> parallel: Warning: No more processes: Decreasing number of running
>> jobs to 4. Raising ulimit -u or /etc/security/limits.conf may help.
>> ...
>>
>> Continously outputs warnings and doesn't finish in a timely manor,
>> because of a line length error on the remote ssh's startup.
>> Running -N 1000 no Warning's and works fine.
>>