Re: [Qemu-devel] [RFC 0/1] Add live migration support to the PVRDMA device

[Dmitry Fleytman]

> On 25 Jun 2019, at 11:49, Marcel Apfelbaum <address@hidden> wrote:
> 
> Hi Dmitry,
> 
> On 6/25/19 11:39 AM, Dmitry Fleytman wrote:
>> 
>>> On 25 Jun 2019, at 11:14, Marcel Apfelbaum <address@hidden> wrote:
>>> 
>>> Hi Sukrit
>>> 
>>> On 6/21/19 5:45 PM, Sukrit Bhatnagar wrote:
>>>> Hi,
>>> [...]
>>>> This RFC is meant to request suggestions on the things which are
>>>> working and for help on the things which are not.
>>>> 
>>> [...]
>>>> What is not working:
>>>> 
>>> [...]
>>>> * It seems that vmxnet3 migration itself is not working properly, at least
>>>>   for me. The pvrdma device depends on it, vmxnet3 is function 0 and pvrdma
>>>>   is function 1. This is happening even for a build of unmodified code from
>>>>   the master branch.
>>>>   After migration, the network connectivity is lost at destination.
>>>>   Things are fine at the source before migration.
>>>>   This is the command I am using at src:
>>>> 
>>>>   sudo /home/skrtbhtngr/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
>>>>     -enable-kvm \
>>>>     -m 2G -smp cpus=2 \
>>>>     -hda /home/skrtbhtngr/fedora.img \
>>>>     -netdev tap,id=hostnet0 \
>>>>     -device vmxnet3,netdev=hostnet0,id=net0,mac=52:54:00:99:ff:bc \
>>>>     -monitor telnet:127.0.0.1:4444,server,nowait \
>>>>     -trace events=/home/skrtbhtngr/trace-events \
>>>>     -vnc 0.0.0.0:0
>>>> 
>>>>   Similar command is used for the dest. Currently, I am trying
>>>>   same-host migration for testing purpose, without the pvrdma device.
>>>>   Two tap interfaces, for src and dest were created successfully at
>>>>   the host. Kernel logs:
>>>>   ...
>>>>   br0: port 2(tap0) entered forwarding state
>>>>   ...
>>>>   br0: port 3(tap1) entered forwarding state
>>>> 
>>>>   tcpdump at the dest reports only outgoing ARP packets, which ask
>>>>   for gateway: "ARP, Request who-has _gateway tell guest1".
>>>> 
>>>>   Tried using user (slirp) as the network backend, but no luck.
>>>>      Also tried git bisect to find the issue using a working commit (given
>>>>   by Marcel), but it turns out that it is very old and I faced build
>>>>   errors one after another.
>>>> 
>>>>   Please note that e1000 live migration is working fine in the same setup.
>>>> 
>>> I tried to git bisect , but I couldn't find a working version of vmxnet 
>>> supporting live migration ....
>>> I tried even a commit from December 2014 and it didn't work.
>>> 
>>> What is strange (to me) is that the networking packets can't be sent from 
>>> the guest (after migration)
>>> even after rebooting the guest.
>> This makes me think that some network offload configuration wasn’t properly 
>> migrated or applied.
>> What network backend are you using?
> 
> Suktrit tried with tap device, I tried with slirp.
> If you can point me to the property that disables all the offloads it will 
> really help.
> 
>> Do you see any outgoing packets in the sniffer?
> 
> I didn't use the sniffer, I checked dmesg in guest, there was a line 
> complaining that it can't send packets.

I see. If it cannot send packet on the guest side, then it’s not an offload.
A snippet from dmesg will be helpful indeed.

> 
> Thanks,
> Marcel
> 
>>> Any help or pointer would be greatly appreciated.
>>> Thanks,
>>> Marcel
>>> 
>>> 
>>> [...]