Re: [Qemu-devel] vhost-pci and virtio-vhost-user

[Jason Wang]

On 2018年01月12日 13:20, Yang, Zhiyong wrote:
> > >    Both vhost-pci and virtio-vhost-user work using shared memory access
> > > to the guest RAM of the other VM.  Therefore they can poll virtqueues
> > > and avoid vmexit.  They do also support cross-VM interrupts, thanks to
> > > QEMU setting up irqfd/ioeventfd appropriately on the host.
> > >
> > > Stefan
> > So in conclusion, consider the complexity, I would suggest to figure out
> > whether or not this (either vhost-pci or virito-vhost-user) is really required
> > before moving ahead. E.g, for VM2VM direct network path, this looks simply
> > an issue of network topology instead of the problem of device, so there's a
> > lot of trick, for vhost-user one can easily image to write an application (or 
> > use
> > testpmd) to build a zerocopied VM2VM datapath, isn't this not sufficient for
> > the case?
> As far as I know,  dequeue zero copied feature of vhost user PMD can't help 
> improve throughput for small packest ,such as 64 bytes.
> On the contrary, it causes perf drop.  The feature mainly helps large packets 
> throughput.

Can you explain why? And what's the number of:

1) 64B/1500B zerocopy
2) 64B/1500B datacopy
3) 64B/1500B vhost-pci

It makes make feel that vhost-pci is dedicated for small bytes? We 
probably don't want a solution for just a specific size of packets.

>     
> Vhostpci can bring the following advantages compared to traditional solution(vhost/virtio PMD pairs)
> 1.  higher throughput for two VMs. ( Let us see the following  case,  if we use 
> NIC passthrough way to two 2 VMs,  vhostpci RX or TX is handled  running 1 
> single core in VM1,  virtio PMD  is similar on VM2,
> Only RX or TX is handled running on one single core.
> for traditional solution,  except each virtio PMD is running inside each VM,  
> at least one extra core is needed for vhost user RX and TX as an mediator.
> In this case, the bottleneck lies in the two vhost user ports running on one 
> single core, which has double workload.

Does this still make sense for packet size other than 64 byte (e.g 1500B)?

>   
>
> 2. Low latencies (have shorter data path than tradition soluton, doesn't need 
> to pass host OS any more by vhost user)

Is this still true if you do busy polling on both sides?

> 3. reduce nearly 50% cores  because  OVS is not involved again if we apply 
> vhostpci/virtio to VMs-chain case.

Well the differences to me is, copy in guest vs copy in host.

- vhost-pci move the copy from host process to pmd in guest, it probably 
save cores but sacrifice the performance of pmd which needs do copy now
- exist OVS may occupy more cores in host, but if saves the ability of 
guest pmd

From the view of performance, it looks to me that copy in host is 
faster since it has less overhead e.g vmexits. Vhost-pci probably needs 
more vcpus to compete with current solution.

Thanks

> Thanks
> Zhiyong