Dear qemu-devel hackers,
I am writing to ask for some technical advice. I am making embarrassingly slow progress on finding a good way to integrate the Snabb Switch user-space ethernet switch (
http://snabb.co/snabbswitch/) with QEMU for efficient ethernet I/O.
Stefan put us onto the highly promising track of vhost/virtio. We have implemented this between Snabb Switch and the Linux kernel, but not directly between Snabb Switch and QEMU guests. The "roadblock" we have hit is embarrasingly basic: QEMU is using user-to-kernel system calls to setup vhost (open /dev/net/tun and /dev/vhost-net, ioctl()s) and I haven't found a good way to map these towards Snabb Switch instead of the kernel.
We have several ideas on the table and we would love some technical feedback on what sounds like a good way forward -- ideally a better alternative that we haven't thought of at all, not being QEMU gurus ourselves.
Here are the ideas on the table right now:
1. Use FUSE to implement a complete clone of /dev/net/tun and /dev/vhost-net inside Snabb Switch. Implement every ioctl() that QEMU requires.
2. Extend QEMU to support a user-user IPC mode of vhost. In this mode QEMU would not use ioctl() etc but some other system calls that are appropriate for IPC between user-space processes.
3. Use the kernel as a middle-man. Create a double-ended "veth" interface and have Snabb Switch and QEMU each open a PF_PACKET socket and accelerate it with VHOST_NET.
#1 is appealing _if_ it can really be done. Risk is that we hit a road-block when implementing the behavior of the ioctl()s, for example have trouble mapping guest memory or getting hold of an eventfd, and that FUSE is kinda heavy-weight.
#2 is appealing _if_ it can be done in a nice way. I don't know which system calls would be appropriate and I don't know how to write the code inside QEMU in a neat way.
#3 is appealing _if_ there is no significant overhead e.g. an extra memory copy inside the kernel, or if it's really quick to do as a temporary stop-gap.
We would love some words of wisdom about the options above and/or a new idea :-)
Cheers,
-Luke