Re: [Qemu-devel] [RFC PATCH RDMA support v5: 03/12] comprehensive protocol documentation

[Michael R. Hines]

You cannot write data in the pipeline because you do not have the
permissions to do so yet until the registrations in the pipeline have
completed and been received by the primary VM.

On 04/11/2013 10:50 AM, Paolo Bonzini wrote:
> Il 11/04/2013 16:37, Michael S. Tsirkin ha scritto:
> > pg1 ->  pin -> req -> res -> rdma -> done
> >          pg2 ->  pin -> req -> res -> rdma -> done
> >                  pg3 -> pin -> req -> res -> rdma -> done
> >                         pg4 -> pin -> req -> res -> rdma -> done
> >                                pg4 -> pin -> req -> res -> rdma -> done
> >
> > It's like a assembly line see?  So while software does the registration
> > roundtrip dance, hardware is processing rdma requests for previous
> > chunks.
> Does this only affects the implementation, or also the wire protocol?
> Does the destination have to be aware that the source is doing pipelining?
>
> Paolo

Yes, the destination has to be aware. The destination has to acknowledge
all of the registrations in the pipeline *and* the primary-VM has to block
until all the registrations in the pipeline have been received.