Re: [Qemu-devel] [PATCH v2 0/5] Add feature to start QEMU without vhost-user backend

[Tetsuya Mukawa]

On 2015/06/22 17:14, Michael S. Tsirkin wrote:
> On Mon, Jun 22, 2015 at 12:50:43PM +0900, Tetsuya Mukawa wrote:
>> Hi guys,
>>
>> Here are patches to add feature to start QEMU without vhost-user backend.
>> Currently, if we want to use vhost-user backend, the backend must start 
>> before
>> QEMU.
> Aren't you looking for something like xinetd?

It will help for restricting starting order, but not help for
reconnection of vhost-user backend.

>> Also, if QEMU or the backend is closed unexpectedly, there is no way to
>> recover without restarting both applications.
> This was previously discussed:
> https://lists.gnu.org/archive/html/qemu-devel/2014-03/msg00585.html
> It doesn't look like any of the issues have been addressed.
>
> In my opinion, the right way to handle this requires
> guest cooperation.  For example, we can extend virtio 1 to allow a
> single queue to reset.
>
> We then can do something like the following:
> - hypervisor detects queue backend disconnect.
> - hypervisor sets flag and notifies guest
> - guest resets queue, optionally discarding queued packets
> - hypervisor detects (or requests if it's a client), backend reconnect
> - hypervisor detects guest queue reset
> - hypervisor notifies guests about backend reconnect
> - hypervisor sends hypervisor device state (e.g. queue addresses and
>   indices) to backend
>
> Reconnect isn't robust without such an extension.

I appreciate your all comments. It's truly helpful.
Do you have any plans to support such queue reset features in virtio spec?
Also, I haven't known following behavior.

> some guests crash if we never complete handling buffers,
> or networking breaks, etc ...

Does it mean that device needs to return buffers through used ring as
soon as possible?
If so, when backend switch is under heavy load, some guests could be
crashed?

>> Practically, it's not useful.
> One thing we do seem to lack is specifying an action
> taken on disconnect. It could be an action similar to r/werror.
> One extra useful option could be to exit qemu.

My implementation treats disconnection as link down.
And when backend is connected, link status will be changed.
So here is behavior of my patch.

- hypervisor detects queue backend disconnect.
- hypervisor sets link status down and notifies guest
- optionally, guest stops sending and receiving packets
  => As you mentioned above, this step may crash some guests.
- hypervisor detects (or requests if it's a client), backend reconnect
- hypervisor make sure backend supports features that the former backend
supports.
- hypervisor sends hypervisor device state (e.g. queue addresses and
  indices) to backend
- backend reads virtqueues, and recover correct indices.
  => Some backends needs to have this feature to reconnect correctly.
- hypervisor notifies guests about backend reconnect as link status up

>> This patch series adds following features.
>>  - QEMU can start before the backend.
> I don't see the point of this one.

Sorry for my bad English. Here is correct.
 - Guest can start before the backend.

Currently, guest will not start without vhost-user backend application
when vhost-user is specified.

>>  - QEMU or the backend can restart anytime.
>>    connectivity will be recovered automatically, when app starts again.
>>    (if QEMU is server, QEMU just wait reconnection)
>>    while lost connection, link status of virtio-net device is down,
>>    so virtio-net driver on the guest can know it
>>
>> To work like above, the patch introduces flags to specify features vhost-user
>> backend will support.
>>
>> Here are examples.
>> ('backend_features' is the new flags. Each bit of the flag represents
>> VIRTIO_NET_F_* in linux/virtio_net.h)
>>
>>     * QEMU is configured as vhost-user client.
>>      -chardev socket,id=chr0,path=/tmp/sock,reconnect=3 \
>>      -netdev 
>> vhost-user,id=net0,chardev=chr0,vhostforce,backend-features=0x68000 \
>>      -device virtio-net-pci,netdev=net0 \
>>
>>     * QEMU is configured as vhost-user server.
>>      -chardev socket,id=chr0,path=/tmp/sock,server,nowait \
>>      -netdev 
>> vhost-user,id=net0,chardev=chr0,vhostforce,backend-features=0x68000 \
>>      -device virtio-net-pci,netdev=net0 \
> This relies on management knowing the feature bitmap encoding of the
> backend, not sure how practical that is.

If we can assume which vhost-user backend application is used, we can
specify backend-features value like above.
Probably such an assumption is not so strange in practice.
And above option allow us to start VM before starting backend application.

I guess it's useful, but if you don't think, I could remove this feature.
In the case, still VM needs to start before backend.
And when backend is connected, virtio-net device will keep backend
features for future reconnection.
(When backend reconnects again, device will make sure that backend
supports features that the former backend supports.)

Regards,
Tetsuya