[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] [RFC] Extend concurrency support
From: |
Daniel Stenberg |
Subject: |
Re: [Bug-wget] [RFC] Extend concurrency support |
Date: |
Tue, 20 May 2014 20:14:56 +0200 (CEST) |
User-agent: |
Alpine 2.00 (DEB 1167 2008-08-23) |
On Tue, 20 May 2014, Tim Ruehsen wrote:
Not sure what other people think about it, but I think wget2, whatever it
will be, should be based on libcurl and focus the wget development on what
wget does better, eg recursive downloads.
Libcurl is one option (and not the worst). At least it would replace the
HTTP and FTP send and receive (plus the underlying TCP network handling -
what about DNS caching ?). This is just a small amount of Wget's code to
replace.
I'll start my response by clearly spell out that I am the libcurl maintainer
and primary developer. I'm biased as hell.
FTP, HTTP, TLS and sockets seem to be about 25% of wget code (very roughly
counted). Replacing the network layers with libcurl would not remove those 25%
completely, as I assume there would have to be adaptions and stuff written
anyway. Also, not everything would be provide exactly in way wget would prefer
it since the designs are quite different.
The benefit would not just be less code in wget. libcurl offers a substantial
amount of more functionality in the network layer than what wget has. And yes,
libcurl has a DNS cache.
In the past I've been told arguments such that the licensing of libcurl and
the fact that libcurl is not a GNU project to be blocking reasons against
using it in wget. I don't know if there's any rules or guidelines that
actually dictate this, but those libcurl facts won't change.
In my eyes, the biggest drawback with switching to libcurl is that it'll
require quite a big ripping-out-and-replacing-the-carpet-beneath-us operation,
and one that'll require one or more dedicated contributors to do some heavy
lifting to get everything on track.
- Cookie logic (incl. public suffix handling)
libcurl also provides cookie support.
- threading abstraction API
libcurl offers parallel transfers without the use of threads so wget actually
wouldn't need threads if based on libcurl. At least not for that reason.
All this said, I won't be the person who'd do the big part of the work so I
won't tell you which way to go. Of course I'll respect whatever decision
that's made and I'll help out with my little bits whichever the decision is
and however the future turns.
--
/ daniel.haxx.se
Re: [Bug-wget] [RFC] Extend concurrency support, Ángel González, 2014/05/21