Re: gnulib broken on systems lacking fchdir

[Matthew Woehlke]

Eric Blake wrote:
> Matthew Woehlke writes:
> > That sounds like a good idea, but... does that mean I have to *write* an 
> > entire unistd.h *and* make it work everywhere, or is there a way to 
> > 'drop in' one that pulls the system unistd.h, plus extras?
>
> For an example of how to provide a replacement <unistd.h>, see how gnulib 
> already provides a replacement <sys/stat.h> which pulls in the system version, 
> then touches it up as needed.  You would really be writing lib/unistd_.h, which 
> first includes @ABSOLUTE_UNISTD_H@, then, if HAVE_FCHDIR is not defined, 
> replaces fchdir with rpl_fchdir, etc.

Thanks for the info. Actually, I found unistd_safer.h too, which might 
help? :-)

> > As mentioned, so far this has nothing to do with coreutils except that I 
> > know it *will* affect coreutils. Right now I'm worrying about gzip.  
> > But... I'm also planning on building gettext (albeit not a version that 
> > has the newer *at stuff AFAIK).
>
> To my knowledge, gettext does not depend on fchdir (as evidenced by the fact 
> that it builds on mingw).  But coreutils, findutils, tar, and gzip all use 
> gnulib directory traversal.

Ok, but I may try to avoid GPL dependencies anyway.

> > Back to the technical standpoint, come to think of it don't most systems 
> > limit # of fd's to a reasonable number like 1024? 
>
> No, the GNU spirit is to avoid arbitrary limits, and 1024 is arbitrary
> [snip]

No, no, I would never *write* code with such a limit :-), I was just 
observing that the likelihood of having so many fd's might make a 
dynamically allocated flat array an acceptable solution (since 99.9% 
likely it will be 1024-or-fewer entries in size).

> Instead, I would make it a malloc'd 
> array, perhaps defaulting to something like 16 elements
> [snip]

Er, right. That's what I was thinking anyway. :-)

> And remember, the array is not 
> likely to be sparse, since open() is required to return the lowest-available fd 
> (and even if the *-safer modules are in effect, you still get the lowest-
> available fd greater than 2).

Ah, yes, well. My point about sparsity was mostly that I expect it to be 
relatively sparse if for some reason an fd of, say, 87649813 shows up. :-)

> As for an audit of uses of fchdir in gnulib, here is my preliminary analysis:

Wow, didn't ask for that, but thanks! Anyway, no time to continue this 
tonight, will look at it tomorrow or next week. Appreciate the help!

> at-func.c and openat.c use fchdir on arbitrary fds.  It would take an audit of 
> coreutils to see if it uses any of the *at functions raw, or whether it always 
> goes through the fts interface; depending on that audit, you may have to 
> override fileno, fopen, etc.

Actually, in at least glib, I saw 0 occurrences of fileno, so... :-) 
Besides, I think if I have to override fopen that's going to get really 
ugly, so probably for now I will audit coreutils as well, and hopefully 
write an implementation that only wraps open() and friends. If someone 
down the line needs the fchdir module and is passing FILE*-derived fd's, 
I'll let them do that work.

> For that matter, if you implement fchdir, then save-cwd.c can be simplified, 
> since it is currently catering to platforms that lack fchdir.

Sweet. :-)

--
Matthew
"Lost a planet, Obi Wan has? How embarrassing..."
 -- Yoda (Star Wars II: Attack of the Clones)