Re: I: coreutils tests/misc/nproc-avail fails on GNU/Linux without /proc and /sys mounted

[Pádraig Brady]

On 10/01/10 01:14, Giuseppe Scrivano wrote:
> when --all fails for any reason, I think we should try with the number
> of available processing units, at least it is a more accurate value than
> return 1 (and document this behaviour).
>
> Bruno, Jim, what do you think?

Just to summarize what's happening here...

There are 3 CPU counts possible:

total >= online >= available

"online" corresponds to the CPUs enabled system wide,
whereas "available" is what's available to a particular
process which may be less due to affinity config.

"online" is not currently exposed through nproc
but for reference is got on linux using:
$ strace -e open getconf _NPROCESSORS_ONLN
  open("/proc/stat"
    for_each_online_cpu() //available to scheduler

"available" on linux is determined using:
nproc
  glibc::sched_getaffinity()
    return corresponding_syscall();

"total" on linux is determined using:
nproc --all
  glibc::sysconf(_NPROCESSORS_CONF)
    __get_nprocs_conf ()
    {
      int result = 1;
      /* XXX Here will come a test for the new system call.  */
      if (open("/sys/devices/system/cpu"))
        result = parse();
      else if (open("/proc/cpuinfo"))
        result = parse();
      return result;
    }

The last one above is giving the issue as
it relies on /sys or /proc being available.
The XXX comment above suggests that there might
be a new syscall to get the total count, but
`man syscalls` doesn't show anything obvious to me.
Now we don't get a failure from sysconf(_NPROCESSORS_CONF),
so really that needs to be fixed. I think in the unusual
case where one can't open /proc/ or /sys/ that it should
try sched_getaffinity() as that will be at least as
accurate as hard coding "1" and will maintain the
expected count ordering above.
I will log a bug against glibc for this.

So what about a possible work around?
How about doing this in nproc(1):

if (num_processors(NPROC_ALL) == 1)
    return num_processors(NPROC_CURRENT_OVERRIDABLE)

That would of course do a redundant syscall in certain cases
but that might be OK actually as relative to the cost
of a process it's not much, and --all wouldn't
be the usual usage anyway.

The alternative of adding skips to the test seem
a bit more messy to me.

Proposed fix is attached.

cheers,
Pádraig.

nproc-avail-fail.diff
Description: Text Data