[Bug-wget] Strange wget behavior when saving to stdout

[Micah Cowan]

The current development sources, when invoked like:

  wget -O - micah.cowan.name >/dev/null

Provides output like:

--2012-06-15 17:40:29--  http://micah.cowan.name/
Resolving micah.cowan.name (micah.cowan.name)... 174.136.4.17
Connecting to micah.cowan.name (micah.cowan.name)|174.136.4.17|:80...
connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “STDOUT”

    [ <=>                                                          ]
78,473       385KB/s   in 0.2s

2012-06-15 17:40:30 (385 KB/s) - written to stdout [78473]

wget: write error: No such file or directory
$

The tail end of strace has:

write(2, "2012-06-15 17:20:24 (215 KB/s) -"..., 602012-06-15 17:20:24
(215 KB/s) - written to stdout [78473]

) = 60
close(1)                                = 0
munmap(0x7f8738708000, 4096)            = 0
open("/usr/share/locale-langpack/en/LC_MESSAGES/wget.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, "wget: ", 6wget: )                   = 6
write(2, "write error", 11write error)             = 11
open("/usr/share/locale/address@hidden/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale-langpack/address@hidden/LC_MESSAGES/libc.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
)                       = 1
exit_group(1)                           = ?
$

At first, I assumed wget was using errno improperly. Imagine my
surprise, though, when running wget under a debugger, to find that at
the tail end of main(), exit() gets called with an argument value of 0.
But gdb then still reports a non-zero exit status - and this "wget:
write error: No such file or directory" seems to get called _after_ wget
calls exit.

I finally figured out what's happening by setting a breakpoint at exit,
and then setting one on write(), _after_ that first point hits.

The backtrace I got back in gdb was:

#0  0x00007ffff71f40f0 in write () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff7188313 in _IO_file_write () from
/lib/x86_64-linux-gnu/libc.so.6
#2  0x00007ffff71881da in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007ffff7188c8e in _IO_file_xsputn () from
/lib/x86_64-linux-gnu/libc.so.6
#4  0x00007ffff715c016 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x00007ffff7156c1e in vfprintf () from /lib/x86_64-linux-gnu/libc.so.6
#6  0x00007ffff717c8e2 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x00007ffff71ff6aa in error () from /lib/x86_64-linux-gnu/libc.so.6
#8  0x0000000000442469 in close_stdout () at closeout.c:117
#9  0x00007ffff714a921 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#10 0x00007ffff714a9a5 in exit () from /lib/x86_64-linux-gnu/libc.so.6
#11 0x0000000000429b8a in main (argc=3, argv=0x7fffffffdce8) at main.c:1642

The close_stdout() thing from closeout.c is from gnulib. I'm going to
leave it to the you, Giuseppe, to find out what's going on beyond
that... IIRC, gnulib offers facilities for excluding certain modules, so
that might be in order here... if it turns out to be gnulib's fault and
not wget (which would be my strong suspicion), you may wish to report it
to them as well. I don't know whether wget does something like closing
stdout early or some such, but I'd certainly take a dim view to helper
utility libraries fucking up my program exit - especially in such a
relatively hard-to-debug fashion.

It's things like this, though, that were the reason why I had chosen to
leave gnulib out of the bootstrapping process, and check in gnulib
sources directly, at intervals. Gnulib is a highly unstable bit of code,
and tends to break/change behaviors with great frequency. I never had
any desire to have to debug gnulib around important relase times... :)

.

Note: while I was debugging this problem, I also noticed at least one
call to stat("-"...) in the strace output. Reading the code, I don't
really see where this could happen without certain options (such as -c
or -N) being in place... nevertheless, I think more should be done to
ensure, even with those options, that stat is never called in that way -
all it would take is the existence of a file, ./-, to screw up wget's
operation while writing to stdout, I'm guessing.

Anyway, good luck!
-mjc