[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR)
From: |
John Gilmore |
Subject: |
Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR) |
Date: |
Tue, 21 Jun 2005 16:10:35 -0700 |
I'm picking up the Gigabit Ethernet conversation from a few weeks ago.
Harald Welte said:
> I'm actually not referring to the Ethernet hardware side, but to the
> software stack. Unless you want to write a special 'ursp network stack'
> that sits directly on top of the hardware driver (like PF_RING, but
> that's read only), you will go through the whole 'normal' Linux network
> stack.
>
> For every packet you process, there's an enormous amount of overhead
> you're going through, even if you choose to not use IP and play some
> tricks with raw sockets. Due to the sockets-based interface, you will
> have additional copying of the data.
>
> I'm by no means arguing that the linux network stack is slow. It's just
> not intended for an application which just wants to get big data streams
> with low latency and little overhead into userspace.
I'm surprised to hear you suggest this (though I note you didn't give
numbers, just generalizations like "enormous"). The first time I
looked seriously at a network stack was in a Van Jacobson/Mike Karels
class about how they made BSD TCP on Sun-3/50s run at 1 MByte/sec,
within a few % of the theoretical maximum thruput of 10Mbit/s
Ethernet. The last time I looked at the Linux network stack was when
Dave Miller was cutting his teeth on the SPARC Linux port and making
100MB/s Ethernet run within a few % of theoretical maximum (he quit
when he exceeded 10MBytes/sec over TCP from user process to user
process). I had of course expected that somebody would do the same
thing for Gigabit Ethernet. This appears to have been done; GigE
cards seem to be able to push large amounts of data under Linux.
What thruput do YOU see using a good GigE card under Linux? Can you move
100 Mbytes/sec from user process to user process with TCP, across two GigE
interfaces and a GigE switch? If so, it's doing much better than USB2
ever will, and much better than any version of FireWire.
Here's a msg that says Linux was getting 100MBps (presumably MBytes/s)
on 2.6.0-test2 in 2003:
http://www.uwsg.iu.edu/hypermail/linux/net/0308.1/0037.html
Here's one that says 930 Mbps using 2.4.21 on a BiXeon in 2004:
http://oss.sgi.com/projects/netdev/archive/2004-04/msg00474.html
Here's one that shows 870 Mbits/s in 2002 on 2.4.18 (long haul from
Sunnyvale to Chicago):
http://www-iepm.slac.stanford.edu/monitoring/bulk/fast/
This msg says that with a 10GB/s Ethernet card (note: 10x as fast as GigE)
they can send UDP at 1.9GBps, and send TCP at 2.2Gbps from Chicago to
Switzerland. Clearly the Linux kernel overhead isn't keeping thruput
below a gigabit:
http://www-iepm.slac.stanford.edu/monitoring/bulk/10ge/
There is a lot of research about how to make TCP run at gigabit speeds
over *long distances*, particularly in recovering from dropped packets.
Nobody seems to need to do research on how to make TCP run at gigabit
speeds across the room; it already works. As does UDP.
Harald, in what way is your experience different?
John
- GigE (was Re: [Discuss-gnuradio] DSP based SDR), Harald Welte, 2005/06/06
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Darrell Harmon, 2005/06/06
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR),
John Gilmore <=
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Harald Welte, 2005/06/22
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Marcus D. Leech, 2005/06/22
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Stephane Fillod, 2005/06/22
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), David Young, 2005/06/22
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Eric Blossom, 2005/06/23
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Harald Welte, 2005/06/24
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Eric Blossom, 2005/06/24
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), David Carr, 2005/06/24
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), Philip Balister, 2005/06/24
- Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR), John Gilmore, 2005/06/24