[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Bug in version 20170322: certain functions in the Bash environment b
From: |
Michael Klement |
Subject: |
Re: Bug in version 20170322: certain functions in the Bash environment break invocation of env_parallel on macOS 10.12.4 |
Date: |
Sun, 16 Apr 2017 10:00:52 -0400 |
Thanks for the detailed response.
Yes, a specific error message might be helpful.
I can confirm that the --env approach is effective as a workaround in my case.
> As the above is the recommended usage, maybe --env _ should be default
> if ~/.parallel/ignored-vars exist?
That might be a little obscure.
I think if the environment problem can be reliably detected, then the
associated error message could recommend the --env approach as an alternative.
Michael
> On Apr 16, 2017, at 8:57 AM, Ole Tange <ole@tange.dk> wrote:
>
> On Sat, Apr 15, 2017 at 3:06 PM, Michael Klement
> <michael.klement@usa.net> wrote:
>
>> I'm aware of that workaround, but I thought you'd want to know that
>> env_parallel is not fully robust and fails in obscure ways.
>> Is env_parallel deprecated? The man page doesn't seem to say so.
>
> env_parallel is pretty new and thus not the same production quality as
> parallel.
>
>> 2017-04-14 20:53 GMT+02:00 Michael Klement <michael.klement@usa.net>:
> :
>>> v3.2.57: only stderr output:
>>>
>>> -bash: /usr/bin/which: Argument list too long
>>> -bash: -n: command not found
>
> Given your attached file, you are most likely being hit by the
> environment being too big:
>
> From man env_parallel:
>
> Environment space
> By default env_parallel will export all environment
> variables, arrays, aliases, functions and shell options
> (see details for the individual shells below).
>
> But this only works if the size of the current environment
> is smaller than the maximal length of a command and
> smaller than half of the max if running remotely. E.g. The
> max size of Bash's command is 128 KB, so env_parallel will
> fail if set | wc -c is bigger than 128 KB. Technically the
> limit is in execve(1) which IPC::open3 uses.
>
> Bash completion functions are well-known for taking up
> well over 128 KB of environment space and the primary
> reason for causing env_parallel to fail.
>
> Instead you can use --env to specify which variables,
> arrays, aliases and functions to export as this will only
> export those with the given name.
>
> And earlier:
>
> Due to the problem with environment space (see below)
> the recommended usage is:
>
> # Run once to record the "empty" environment
> env_parallel --record-env
> alias myalias=echo
> myvar=it
> myfunc() { myalias $1 $myvar works.; }
>
> # Use --env _ to only transfer the names not in the "empty" environment
> env_parallel --env _ -S localhost myfunc ::: Yay,
>
> But even tough it is documented your bug report is valuable:
> env_parallel should detect this error and give you a sensible error
> message instead.
>
> As the above is the recommended usage, maybe --env _ should be default
> if ~/.parallel/ignored-vars exist?
>
>
> /Ole