Il 14/02/2013 20:29, Michael R. Hines ha scritto:
Are you still using the tcp for transferring device state? If so you
can call the tcp functions from the migration rdma code as a first
step but I would prefer it to use RDMA too.
This is the crux of the problem of using RDMA for migration: Currently
all of the QEMU migration control logic and device state goes through
the the QEMUFile implementation. RDMA, however is by nature a zero-copy
solution and is incompatible with QEMUFile.
With the patches I sent yesterday, there is no more buffering involved
in migration.c. All data goes straight from arch_init.c to a QEMUFile.
QEMUFile still does some buffering, but this should change with other
patches that Orit is working on.
Using RDMA for transferring device state is not recommended: Setuping an
RDMA requires registering the memory locations on both sides with the
RDMA hardware. This is not ideal because this would require pinning the
memory holding the device state and then issuing the RDMA transfer for
*each* type of device - which would require changing the control path of
every type of migrated device in QEMU.
Yes, this would not work well. However, you can (I think) define a
QEMUFileOps implementation for RDMA that:
1) registers the buffer of a QEMUFile with the RDMA hardware;
2) in its get_buffer (receive) and put_buffer (send) callbacks, issues a
synchronous RDMA transfer;
3) unregisters the buffer in the close callback.
As a proof of concept, this would also work (though it would make no
sense) for transferring the RAM; in the end of course it would be used
only for the device state.
It's not a problem to add more operations to QEMUFileOps or similar.
It's also not a problem to change the way buf is allocated, if you need
it to be page-aligned or something like that.
It is much better than adding migration_use_rdma() everywhere.
Paolo