Re: 'wait -n' with and without id arguments

[Chet Ramey]

On 9/8/24 8:35 PM, Zachary Santer wrote:

This is still a discussion about interactive shell behavior only.

> > What behavior do you want from the command lists that differs from what I
> > described above? Since shell functions are essentially lists, you should
> > get the same behavior from both.
>
> You'd have to restrict job status notification to only ever occur
> immediately prior to a prompt, in both posix and default mode, and
> then you'd still need a blurb in the BUGS section of the manual saying
> that 'set -b' has a potentially surprising impact on 'wait -n' in the
> interactive shell.

This is pretty much posix mode, and there's no reason to list this as a
bug when it's not.

> For this to work, you'd have to choose one of the following new behaviors:
>
> 1) Background jobs that are both forked and cleared from the jobs
> table by a call to 'wait' in the time between an accept-line and the
> following prompt would never receive a job id or be notified to the
> user in any way. 

That's not how job control works. Jobs are created and job numbers assigned
when the background process is created.

> 2) Job ids assigned to background jobs continue to increase
> monotonically, between accept-line and prompt, even as some of those
> jobs are removed from the jobs table by calls to 'wait'.

No shell works this way, and there's not a good reason to adopt it.

> If the 'jobs' builtin is called in the midst of a command list being
> run with either behavior, this would cause the same updates to the
> jobs table and list of saved pids and statuses as would occur
> immediately prior to a prompt.

So you are saying that prompt notifications and `jobs' have the same
effect. POSIX implies but does not require this, and there is differing
behavior among current implementatations.


> The user would have to know that
> calling the 'jobs' builtin would have an impact on what processes
> 'wait -n' without id arguments will return the termination status of.
> That would have to be documented in the man page.

This is posix mode.



> > > However, I think the benefit to
> > > consistent behavior far outweighs the hardship caused to whoever would
> > > write a script intended for use within the interactive shell that depends
> > > on 'wait -n' without id arguments ignoring background processes that the
> > > user has already been notified of via the 'jobs' output.
> >
> > Consider programmable completion frameworks, commands executed via
> > `bind -x', or traps (e.g., DEBUG) intended to provide enhancements to
> > the standard behavior, all of which exist and have generated reports or
> > requests for features.
> >
> > People put pretty complicated stuff into PROMPT_COMMAND and other prompts,
> > too.
> >
> > We don't know how the existing uses would be affected by changes until I
> > make them.
>
> Could changes to job status notifications cause issues for these users as well?

You tell me.

> Generally speaking, I would expect most functions defined in any of
> the above that call 'wait' or 'wait -n' from an interactive
> environment to track and use explicit pid arguments, to avoid waiting
> on other background jobs the user forked themselves. 

It's an assumption. It might be valid.

> In that case, the
> behavior they would see, using 'wait -n', has already changed for the
> better. The use of 'wait -n' without pid arguments in an interactive
> environment is more likely to be something that a user just typed on
> the command line themselves.

Why would a user do this? What's the use case for doing that in an
interactive shell? Not that it really matters.

> > > If the behavior here isn't modified, the man page really should note that
> > > 'wait -n' without id arguments won't return the termination status of a
> > > child process that has already been notified through the 'jobs' output.
> >
> > That is exactly the behavior posix seems to require (`wait -n' aside, but
> > see below): once you notify the user, you delete the job and it disappears
> > forever.
>
> Should still be in the man page. Very few shell programmers are
> reading the POSIX standard.

There is a posix mode section in bash.info.

> Does POSIX provide a rationale for this requirement? 

Look at the descriptions of `jobs' and `wait', which say this explicitly.

> I'd be curious to know if the Austin Group
> people have considered the implications of a feature like 'wait -n'.

Doubtful they considered it explicitly, since the role of a standard is
to standardize existing implementations.


> If you go the route of changing job notification behavior, would that
> be the end of the list of saved pids and statuses? Maintaining that
> list is more useful than simply following POSIX to a tee. What would
> be the benefit to the user of making the termination status of
> notified jobs unavailable to the 'wait' builtin?

I'm not talking about changing default mode, just making bash conform
to the new requirements in the latest POSIX revision. These requirements
happen to align well with your use case.

For instance, with the latest devel branch build:

$ set -o posix
$ sleep 2 &
[1] 20565
$
[1]+  Done                       sleep 2
$ wait 20565
$ echo $?
0
$ wait 20565
bash: wait: pid 20565 is not a child of this shell
$ echo $?
127
$

This is what you refer to below.

> 'wait -n' with pid arguments now has access to this list, which is
> good. It wouldn't be going much further to allow 'wait -n' without pid
> arguments to act on the list as well.

Well, you'd either have to arrange things so the user doesn't get the same
pid and status returned multiple times -- by removing it from this list or
some other mechanism. Since that's what happens in posix mode, it looks
like posix mode fits your use case here.

> Meanwhile, POSIX is telling you
> the list shouldn't even exist to begin with.

POSIX is ambiguous about whether or not notifications before prompting
remove jobs from the jobs list, so you still have to keep up to CHILD_MAX
statuses until the user runs `jobs' or `wait'. How you choose to do that
is up to the implementation. Back in the day, I didn't want the jobs table
to keep potentially thousands of terminated jobs, so I chose a lighter
weight way to do it. The ability to wait for a process multiple times in
default mode was a side benefit.

> Given some very simplistic testing, I see that 'wait' in both mksh and
> zsh will return the termination status of a background job that has
> already been notified to the user as terminated in the interactive
> shell, once. Subsequent calls to 'wait' with that same pid argument
> return status 127.

Yes, there are differences between implementations here and POSIX is
ambiguous about what it requires as a result. The NetBSD sh, for instance,
returns 127 for any wait call after the before-prompt notification, and I
know kre has very definite ideas about what POSIX requires, or should
require.

> Making 'wait -n' without id arguments work the way I'd like it to
> would likely entail that 'wait' with id argument would work like it
> does in mksh and zsh.

You want posix mode.

> It would be no good for 'wait -n' without id
> argument to continue to return the termination status of the same
> terminated background job over and over again. With access to the list
> of pids and statuses, it would have to remove the element for the
> terminated background job it reports for it to not simply report it
> again upon the next invocation. It would be more consistent for 'wait'
> with pid argument to then also remove the element from this list for
> the id argument it was passed.

Right: posix mode.


> Getting back to the POSIX point, if bash, mksh, and zsh all violate
> POSIX in this way, maybe it's POSIX that should change.

What are the posix violations?

https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html#tag_19_11

is ambiguous.


> Would making job status notifications only ever occur immediately
> prior to a prompt (or when calling 'jobs') violate POSIX too? 

What? No, that's what POSIX says to do, and it's spelled out in the latest
revision, which makes it a new requirement on implementations.


> > What was your initial understanding?
>
> No job status notification at any point except immediately prior to a
> prompt (or when calling 'jobs').

That is posix mode, as clarified in the latest revision. `set -b', of
course, changes the conditions.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/