inifite loop in recent mgetgroups.c:mgetgroups on Darwin/MacOS

[Steven Parkes]

Darwin (and I think other OSes using old NetBSD getgrouplist.c) cause recent
mgetgroups() to loop infinitely. 

The Darwin implementation of getgrouplist doesn't update the ngroups in/out
parameter to getgrouplist correctly and as a result the new implementation
of mgetgroups (as of
http://lists.gnu.org/archive/html/bug-coreutils/2009-01/msg00121.html) loops
infinitely.

In the old code, first it looked at max_n_groups and assumed that if it was
less than the initial value, it had all the groups. In the new code, it
looks at the return code from getgrouplist and assumes that if the call
returns -1, that max_n_groups has been increased. On a working getgrouplist,
there should be no difference in behavior. On Darwin, there is, resulting in
an infinite loop.

I can imagine different ways of patching this but I don't really know what's
preferable on a project basis.

This would revert to the old behavior:

diff --git a/gl/lib/mgetgroups.c b/gl/lib/mgetgroups.c
index e697013..1f045b3 100644
--- a/gl/lib/mgetgroups.c
+++ b/gl/lib/mgetgroups.c
@@ -94,7 +94,7 @@ mgetgroups (char const *username, gid_t gid, GETGROUPS_T
**groups)
            }
          g = h;
 
-         if (0 <= ng)
+         if (0 <= ng || max_n_groups <= N_GROUPS_INIT)
            {
              *groups = g;
              return ng;

The old NetBSD code just set the initial value to NGROUPS_MAX+1, which seems
doesn't really work anymore, either, since Darwin can now have more groups
than that (and I think NGROUPS_MAX is depreciated?)

It's not too terribly hard to detect the bug and manually loop, increasing
the buffer size each time, but that seems kinda gross (and would probably
need protection against another infinite loop).

Happy to provide a patch for the preferred way to address ...

Found in the Gentoo-prefix project:
http://bugs.gentoo.org/show_bug.cgi?id=264007