Re: [Qemu-devel] [PATCH v11 14/15] rdma: introduce MIG_STATE_NONE and change MIG_STATE_SETUP state transition

[Juan Quintela]

"Michael R. Hines" <address@hidden> wrote:
> On 06/25/2013 06:13 AM, Paolo Bonzini wrote:
>> Il 25/06/2013 11:49, Juan Quintela ha scritto:
>>> address@hidden wrote:
>>>> From: "Michael R. Hines" <address@hidden>
>>>>
>>>> As described in the previous patch, until now, the MIG_STATE_SETUP
>>>> state was not really a 'formal' state. It has been used as a 'zero' state
>>>> (what we're calling 'NONE' here) and QEMU has been unconditionally 
>>>> transitioning
>>>> into this state when the QMP migration command was called. Instead we want 
>>>> to
>>>> introduce MIG_STATE_NONE, which is our starting state in the state 
>>>> machine, and
>>>> then immediately transition into the MIG_STATE_SETUP state when the QMP 
>>>> migrate
>>>> command is issued.
>>>>
>>>> In order to do this, we must delay the transition into MIG_STATE_ACTIVE 
>>>> until
>>>> later in the migration_thread(). This is done to be able to timestamp the 
>>>> amount of
>>>> time spent in the SETUP state for proper accounting to the user during
>>>> an RDMA migration.
>>>>
>>>> Furthermore, the management software, until now, has never been aware of 
>>>> the
>>>> existence of the SETUP state whatsoever. This must change, because, timing 
>>>> of this
>>>> state implies that the state actually exists.
>>>>
>>>> These two patches cannot be separated because the 'query_migrate' QMP
>>>> switch statement needs to know how to handle this new state transition.
>>>>
>>>> Tested-by: Michael R. Hines <address@hidden>
>>>> Signed-off-by: Michael R. Hines <address@hidden>
>>>> @@ -316,6 +321,7 @@ static void migrate_fd_cancel(MigrationState *s)
>>>>   {
>>>>       DPRINTF("cancelling migration\n");
>>>>   +    migrate_set_state(s, MIG_STATE_SETUP, MIG_STATE_CANCELLED);
>>>>       migrate_set_state(s, MIG_STATE_ACTIVE, MIG_STATE_CANCELLED);
>>>>   }
>>> This chunk is wrong.
>>>
>>> we can call qme_migrate_cancel() at any point,  and it is going to be
>>> called normally from MIG_STATE_ACTIVE.
>>>
>>>      migrate_set_satet(s, s->state,  MIG_STATE_CANCELLED)
>>>
>>> should do the trick.  Or something like that,  what do you think?
>> I don't like the three-arguments migrate_set_state, but I don't have any
>> better idea.
>>
>> With Juan's modification, it is fine (but not reviewed-by me :)).  While
>> you resend, the first 13 patches of v10 can be merged (pull request).
>> You can then rebase the last three on top.
>
> So, I should send a v11 before Juan applies and generates the pull request?
>
> Did I understand correctly?
>
>> Michael, did you look at the "debugging/getting the protocol ready" mode
>> where all pages are unpinned?
>>
>> Paolo
>>
>
>
> We did not come up with a design on how such a mode would be
> exposed to the user. Implementing this in migration-rdma.c is
> just a few lines of code, but without a clear use case in this patch
> series, I've not chosen expose it to the user yet without some kind
> of agreement with you guys.
>
> Recommendations?

Dunno.  I have tried to use the thing:

I am using -incoming x-rdma:deus:4444

(qemu) char device redirected to /dev/pts/2 (label serial0)
librdmacm: Warning: couldn't read ABI version.
librdmacm: Warning: assuming: 4
librdmacm: Fatal: unable to get RDMA device list
RDMA ERROR: could not create rdma event channel
-incoming x-rdma:deus:4444: RDMA ERROR: could not create rdma event channel


I also used "0" instead of deus (the machine name).

The only thing that I did appart from intsalling librdmacm-devel was:


# cat /etc/udev/rules.d/80-infinibad.rules
KERNEL="rdma_cm", NAME="infiniband/%k", MODE="0666"

As per README.  Something else I need to do to use the tcp
implementations?

Later,  Juan.