[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] Bug: resources can't be mirrored via FTP if their name st
From: |
Darshit Shah |
Subject: |
Re: [Bug-wget] Bug: resources can't be mirrored via FTP if their name starts with a space character |
Date: |
Mon, 8 Apr 2019 14:31:13 +0200 |
User-agent: |
NeoMutt/20180716 |
* Tim Rühsen <address@hidden> [190408 09:24]:
> Hello Christian,
>
> please add options '-d -olog --no-remove-listing' and send us (privately
> if you have secrets in the log file) the files 'log' and .listing.
>
> A problem with FTP directory listings is that there is no standard. A
> leading space could easily be part of the formatting which is meant to
> be human readable, not machine readable. Assuming one space as delimiter
> likely is a regression for other users.
>
> In the long term, you should consider HTTPS instead of FTP.
>
Actually, from the provided data, I can already see what is happening and why.
Just as Tim mentioned, the listing file doesn't have a fixed standard and uses
whitespace as a field separator.
Looking at the `.listing` file that Wget downloaded, I can see that there is
just whitespace between the size and the filename. Wget has no way to tell if
the whitespace is a field separator or part of the filename. The correct way to
deal with this would be for your FTP server to generate a listing file similar
to ls. Where, it wraps the filename in quotes in order to show that the
whitespace is indeed a part of the filename
What FTP server are you using?
> Regards, Tim
>
> On 08.04.19 02:13, Christian Rosentreter wrote:
> >
> > Hi there,
> >
> > A small bug I encountered: if any file or directory on a remote machine has
> > pathnames
> > starting with one or more spaces (e.g. " foobar.txt") then Wget fails to
> > mirror those
> > resources via the good old FTP protocol. Tested with '1.18' and up-to-date
> > '1.20.3' on
> > Mac OS X.
> >
> > If required, I could provide a testing FTP account for this particular
> > setup to a
> > Wget developer (privately.)
> >
> >
> > with kind regards,
> > Christian Rosentreter
> >
> >
> >
> >
> > ----
> >
> > Example:
> >
> > wget \
> > --config auth.config \
> > --mirror \
> > --no-host-directories \
> > --append-output wget.log \
> > ftp://wget.annex.binaryriot.org/
> >
> > The test setup on the remote server (up-to-date Ubuntu) looks something
> > like this,
> > output via 'ls -l':
> > test/
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31 '
> > foobar-with-leading-spaces.txt'
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31 ' foo bar with spaces .txt
> > '
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31 foobar-normal.txt
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31
> > 'foobar-with-trailing-spaces.txt '
> > drwxr-xr-x 2 foobar foobar 4096 Apr 8 00:31 ' dir-with-leading-spaces'
> > drwxr-xr-x 2 foobar foobar 4096 Apr 8 00:31 'dir-with-trailing-spaces
> > '
> > drwxr-xr-x 2 foobar foobar 4096 Apr 8 00:31 ' dir with spaces '
> >
> >
> > The content of the locally generated .listing file inside the "test"
> > directory,
> > looks like this:
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31
> > foobar-with-leading-spaces.txt
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31 foo bar with
> > spaces .txt
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31 foobar-normal.txt
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31
> > foobar-with-trailing-spaces.txt
> > drwxr-xr-x 2 foobar foobar 4096 Apr 8 00:31
> > dir-with-leading-spaces
> > drwxr-xr-x 2 foobar foobar 4096 Apr 8 00:31
> > dir-with-trailing-spaces
> > drwxr-xr-x 2 foobar foobar 4096 Apr 8 00:31 dir with
> > spaces
> >
> >
> > The actual locally mirrored directories and files, output via OS X's 'ls -
> > l' look
> > like this (any resource with leading spaces is obviously missing in this
> > local copy
> > now. That's a wee-bit bad for backups when important files can be
> > unexpectedly
> > M.I.A. ;) )
> > test/
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31 foobar-normal.txt
> > -rw-r--r-- 1 foobar foobar 0 Apr 8 00:31
> > foobar-with-trailing-spaces.txt
> > drwxr-xr-x 3 foobar foobar 102 Apr 8 00:32
> > dir-with-trailing-spaces
> >
> >
> > In the log file (via --append-output) various errors like this are
> > generated (the
> > leading spaces are missing), but with the return code of the command
> > wrongly indicating
> > that the full mirror was actually a success in the end:
> > ...
> > No such file 'test/foobar-with-leading-spaces.txt'.
> > No such file 'test/foo bar with spaces .txt '.
> > No such directory 'test/dir-with-leading-spaces'.
> > No such directory 'test/dir with spaces '.
> > ...
> >
> >
> >
>
--
Thanking You,
Darshit Shah
PGP Fingerprint: 7845 120B 07CB D8D6 ECE5 FF2B 2A17 43ED A91A 35B6
signature.asc
Description: PGP signature