|
From: | Michael R. Hines |
Subject: | Re: [Qemu-devel] [RFC PATCH RDMA support v5: 03/12] comprehensive protocol documentation |
Date: | Sun, 14 Apr 2013 21:10:36 -0400 |
User-agent: | Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2 |
On 04/14/2013 05:16 PM, Michael S. Tsirkin wrote:
On Sun, Apr 14, 2013 at 03:43:28PM -0400, Michael R. Hines wrote:On 04/14/2013 02:51 PM, Michael S. Tsirkin wrote:On Sun, Apr 14, 2013 at 10:31:20AM -0400, Michael R. Hines wrote:On 04/14/2013 04:28 AM, Michael S. Tsirkin wrote:On Fri, Apr 12, 2013 at 09:47:08AM -0400, Michael R. Hines wrote:Second, as I've explained, I strongly, strongly disagree with unregistering memory for all of the aforementioned reasons - workloads do not operate in such a manner that they can tolerate memory to be pulled out from underneath them at such fine-grained time scales in the *middle* of a relocation and I will not commit to writing a solution for a problem that doesn't exist.Exactly same thing happens with swap, doesn't it? You are saying workloads simply can not tolerate swap.If you can prove (through some kind of anaylsis) that workloads would benefit from this kind of fine-grained memory overcommit by having cgroups swap out memory to disk underneath them without their permission, I would happily reconsider my position. - MichaelThis has nothing to do with cgroups directly, it's just a way to demonstrate you have a bug.If your datacenter or your cloud or your product does not want to tolerate page registration, then don't use RDMA! The bottom line is: RDMA is useless without page registration. Without it, the performance of it will be crippled. If you define that as a bug, then so be it. - MichaelNo one cares if you do page registration or not. ulimit -l 10g is the problem. You should limit the amount of locked memory. Lots of good research went into making RDMA go fast with limited locked memory, with some success. Search for "registration cache" for example.Patches using such a cache would be welcome. - MichaelAnd when someone writes them one day, we'll have to carry the old code around for interoperability as well. Not pretty. To avoid that, you need to explicitly say in the documenation that it's experimental and unsupported.
That's what protocols are for. As I've already said, I've incorporated this into the design of the protocol already. The protocol already has a field called "repeat" which allows a user to request multiple chunk registrations at the same time.If you insist, I can add a capability / command to the protocol called "unregister chunk",
but I'm not volunteering to implement that command as I don't have any data showing it to be of any value.That would insulate the protocol against any such future "registration cache" design.
- Michael
[Prev in Thread] | Current Thread | [Next in Thread] |