Re: [PATCH] migration/throttle: Make throttle slower at tail stage

[zhukeqian]

Hi, Juan

On 2020/2/14 20:37, Juan Quintela wrote:
> Keqian Zhu <address@hidden> wrote:
>> At the tail stage of throttle, VM is very sensitive to
>> CPU percentage. We just throttle 30% of remaining CPU
>> when throttle is more than 80 percentage.
> 
> Why?
> 
My original idea is that if we throttle a fixed percentage of CPU every time,
then the VM is more and more sensitive to performance decrease.

For example, if the initial throttle is 10% and we throttle 10% every time. At 
the
beginning, the performance changes from 100% to 90%, which makes little effect 
on VM.
However, if the dirty rate is very high and it is not enough even throttle 80%, 
then
the performance changes from 20% to 10%, which half the performance and makes 
heavy
effect on VM.

In the example above, if throttle 85% is enough, then throttle 90% makes 
unnecessary
performance loss on VM. So this is the reason for slowdown throttling when we 
are about
to reach the best throttle.

> If we really think that this is better that current approarch, just do
> this _always_.  And throothre 30% of remaining CPU.  So we go:
> 
> 30%
> 30% + 0.3(70%)
> ...
> 
> Or anything else.
> 

This should not be a new approach, instead it is an optional enhancement to

current approach. However, after my deeper thinking, the way that throttle
30% of remaining CPU is unusual and not suitable. We should use another way
to slowdown the tail stage.

When dirty bytes is is 50% more than the approx. bytes xfer, we start or 
increase
throttling. My idea is that we can calculate the throttle increment expected.
When dirty rate is about to reach the 50% of bandwidth, the throttle increment
expected will smaller than "cpu_throttle_increment" at tail stage.

Maybe the core code likes this:

-static void mig_throttle_guest_down(void)
+static void mig_throttle_guest_down(uint64_t bytes_dirty, uint64_t bytes_xfer)
 {
     MigrationState *s = migrate_get_current();
     uint64_t pct_initial = s->parameters.cpu_throttle_initial;
-    uint64_t pct_icrement = s->parameters.cpu_throttle_increment;
+    uint64_t pct_increment = s->parameters.cpu_throttle_increment;
+    bool pct_tailslow = s->parameters.cpu_throttle_tailslow;
     int pct_max = s->parameters.max_cpu_throttle;

+    uint64_t cpu_throttle_now = cpu_throttle_get_percentage();
+    uint64_t cpu_now, cpu_target, cpu_throttle_expect;
+    uint64_t cpu_throttle_inc;
+
     /* We have not started throttling yet. Let's start it. */
     if (!cpu_throttle_active()) {
         cpu_throttle_set(pct_initial);
     } else {
         /* Throttling already on, just increase the rate */
-        cpu_throttle_set(MIN(cpu_throttle_get_percentage() + pct_icrement,
+        cpu_throttle_inc = pct_increment;
+        if (pct_tailslow) {
+            cpu_now = 100 - cpu_throttle_now;
+            cpu_target = ((bytes_xfer / 2.0) / bytes_dirty) * cpu_now;
+            cpu_throttle_expect = cpu_now - cpu_target;
+            if (cpu_throttle_expect < pct_increment) {
+                cpu_throttle_inc = cpu_throttle_expect;
+            }
+        }
+        cpu_throttle_set(MIN(cpu_throttle_now + cpu_throttle_inc,
                          pct_max));
     }
 }
__________________________________________________________________________

-            if ((rs->num_dirty_pages_period * TARGET_PAGE_SIZE >
-                   (bytes_xfer_now - rs->bytes_xfer_prev) / 2) &&
+            bytes_dirty_period = rs->num_dirty_pages_period * TARGET_PAGE_SIZE;
+            bytes_xfer_period = bytes_xfer_now - rs->bytes_xfer_prev;
+            if ((bytes_dirty_period > bytes_xfer_period / 2) &&
                 (++rs->dirty_rate_high_cnt >= 2)) {
                     trace_migration_throttle();
                     rs->dirty_rate_high_cnt = 0;
-                    mig_throttle_guest_down();
+                    mig_throttle_guest_down(bytes_dirty_period,
+                                            bytes_xfer_period);
             }
> My experience is:
> - you really need to go to very high throothle to make migration happens
>   (more than 70% or so usually).
> - The way that we throotle is not completely exact.
> 
>> This doesn't conflict with cpu_throttle_increment.
>>
>> This may make migration time longer, and is disabled
>> by default.
> 
> 
> What do you think?
> I think that it is better to change method and improve documentation
> that yet adding another parameter.
> 
> Later, Juan.
> 
> 
> .
> 
Thanks,
Keqian