Re: [Bug-wget] GSoC15: Speed up Wget's Download Mechanism

[Eli Zaretskii]

> Date: Thu, 30 Apr 2015 19:30:39 +0200
> From: Gisle Vanem <address@hidden>
> CC: address@hidden
> 
> wget -q -O NUL
>   
> "https://download-installer.cdn.mozilla.net/pub/firefox/releases/37.0.2/win32/en-GB/Firefox
>  Setup 37.0.2.exe"
> 
> results in 9931 DLL attach/detaches!
> 
> For a 40 MByte file that is approx. 1 new thread per 4 kByte read.
> I was thinking that increasing read-buffer would help. But where?
> The code is bit of a mess IMHO. Increasing the Rx buffer in
> fd_read_body() didn't help. Is this the chief in this regard?
> 
> Without getting any numbers, I can see in 'Process Explorer'
> that all those run_with_timeout() calls (and no '-T0') amount
> to some more user+kernel time. I guess using a profiler is next.
> Or maybe someone knows of a Win-program that can report total
> CPU (kernel/user) time from the cmd-line?

'timep' from "Windows System Programming" can (let me know if you want
the source I use).  This is based on average of 2 runs of Wget 1.16.1
running on a 32-bit Windows XP, with a 30 Mbit/sec cable connection:

  real    00h01m56.500s
  user    00h00m00.823s
  sys     00h00m00.355s

And here's the same from a GNU/Linux machine that downloads at 20.7
MB/sec:

  real    0m2.300s
  user    0m1.600s
  sys     0m0.6000s

> BTW. My ISP gives me 25 Mbit/s in and 10 MBit/s out.

See above; removing -q from the command line indicates that the actual
download speed for this file is around 500 KB/sec.