qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [RFC PATCH RDMA support v5: 03/12] comprehensive protoc


From: Michael R. Hines
Subject: Re: [Qemu-devel] [RFC PATCH RDMA support v5: 03/12] comprehensive protocol documentation
Date: Sun, 14 Apr 2013 21:10:36 -0400
User-agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2

On 04/14/2013 05:16 PM, Michael S. Tsirkin wrote:
On Sun, Apr 14, 2013 at 03:43:28PM -0400, Michael R. Hines wrote:
On 04/14/2013 02:51 PM, Michael S. Tsirkin wrote:
On Sun, Apr 14, 2013 at 10:31:20AM -0400, Michael R. Hines wrote:
On 04/14/2013 04:28 AM, Michael S. Tsirkin wrote:
On Fri, Apr 12, 2013 at 09:47:08AM -0400, Michael R. Hines wrote:
Second, as I've explained, I strongly, strongly disagree with unregistering
memory for all of the aforementioned reasons - workloads do not
operate in such a manner that they can tolerate memory to be
pulled out from underneath them at such fine-grained time scales
in the *middle* of a relocation and I will not commit to writing a solution
for a problem that doesn't exist.
Exactly same thing happens with swap, doesn't it?
You are saying workloads simply can not tolerate swap.

If you can prove (through some kind of anaylsis) that workloads
would benefit from this kind of fine-grained memory overcommit
by having cgroups swap out memory to disk underneath them
without their permission, I would happily reconsider my position.

- Michael
This has nothing to do with cgroups directly, it's just a way to
demonstrate you have a bug.

If your datacenter or your cloud or your product does not want to
tolerate page registration, then don't use RDMA!

The bottom line is: RDMA is useless without page registration. Without
it, the performance of it will be crippled. If you define that as a bug,
then so be it.

- Michael
No one cares if you do page registration or not.  ulimit -l 10g is the
problem.  You should limit the amount of locked memory.
Lots of good research went into making RDMA go fast with limited locked
memory, with some success. Search for "registration cache" for example.

Patches using such a cache would be welcome.

- Michael

And when someone writes them one day, we'll have to carry the old code
around for interoperability as well. Not pretty.  To avoid that, you
need to explicitly say in the documenation that it's experimental and
unsupported.


That's what protocols are for.

As I've already said, I've incorporated this into the design of the protocol
already.

The protocol already has a field called "repeat" which allows a user to
request multiple chunk registrations at the same time.

If you insist, I can add a capability / command to the protocol called "unregister chunk",
but I'm not volunteering to implement that command as I don't have any data
showing it to be of any value.

That would insulate the protocol against any such future "registration cache" design.

- Michael





reply via email to

[Prev in Thread] Current Thread [Next in Thread]