Re: [Qemu-devel] [RFC PATCH] Fake machine for scalability testing

[Daniel P. Berrange]

On Thu, Jan 20, 2011 at 01:50:33PM -0600, Anthony Liguori wrote:
> On 01/20/2011 11:12 AM, Markus Armbruster wrote:
> >Anthony Liguori<address@hidden>  writes:
> >
> >>On 01/18/2011 02:16 PM, Markus Armbruster wrote:
> >>>The problem: you want to do serious scalability testing (1000s of VMs)
> >>>of your management stack.  If each guest eats up a few 100MiB and
> >>>competes for CPU, that requires a serious host machine.  Which you don't
> >>>have.  You also don't want to modify the management stack at all, if you
> >>>can help it.
> >>>
> >>>The solution: a perfectly normal-looking QEMU that uses minimal
> >>>resources.  Ability to execute any guest code is strictly optional ;)
> >>>
> >>>New option -fake-machine creates a fake machine incapable of running
> >>>guest code.  Completely compiled out by default, enable with configure
> >>>--enable-fake-machine.
> >>>
> >>>With -fake-machine, CPU use is negligible, and memory use is rather
> >>>modest.
> >>>
> >>>Non-fake VM running F-14 live, right after boot:
> >>>UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
> >>>armbru   15707  2558 53 191837 414388 1 21:05 pts/3    00:00:29 [...]
> >>>
> >>>Same VM -fake-machine, after similar time elapsed:
> >>>UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
> >>>armbru   15742  2558  0 85129  9412   0 21:07 pts/3    00:00:00 [...]
> >>>
> >>>We're using a very similar patch for RHEL scalability testing.
> >>>
> >>Interesting, but:
> >>
> >>  9432 anthony   20   0  153m  14m 5384 S    0  0.2   0:00.22
> >>qemu-system-x86
> >>
> >>That's qemu-system-x86 -m 4
> >Sure you ran qemu-system-x86 -fake-machine?
> 
> No, I didn't try it.  My point was that -m 4 is already pretty small.

One of the core ideas/requirements behind the "fake QEMU" was
that we won't need to modify applications to adjust the command
line arguments in this kind of way. We want all their machine
definition logic to remain unaffected. In fact my original
prototype did not even require addition of the passing of an
extra '-fake-machine' argument, it would have just been a plain
drop in alternative QEMU binary. It also stubbed out much of
the KVM codepaths, so you could "enable"  KVM mode without
actually really having KVM available on the host.

> >>In terms of memory overhead, the largest source is not really going to
> >>be addressed by -fake-machine (l1_phys_map and phys_ram_dirty).
> >git-grep phys_ram_dirty finds nothing.
> 
> Yeah, it's now ram_list[i].phys_dirty.
> 
> l1_phys_map is (sizeof(void *) + sizeof(PhysPageDesc)) * mem_size_in_pages
> 
> phys_dirty is mem_size_in_pages bytes.
> 
> >>I don't really understand the point of not creating a VCPU with KVM.
> >>Is there some type of overhead in doing that?
> >I briefly looked at both main loops, TCG's was the first one I happened
> >to crack, and I didn't feel like doing both then.  If the general
> >approach is okay, I'll gladly investigate how to do it with KVM.
> 
> I guess what I don't understand is why do you need to not run guest
> code?  Specifically, if you remove the following, is it any less
> useful?

IIUC, if you don't have the following change, then the guest CPUs
will actually execute the kernel/bootable disk configured, causing
host CPU utilization to rise. Even if it only adds 2% load on the
host, this quickly becomes an issue as you get 200 or more VMs on
the host. Ideally we would have the main loop completely disabled,
not merely the CPUs, because this would avoid all possible background
CPU load that any QEMU internal timers might cause.

Basically the desired goal is, make no change to the QEMU command
line aguments, but have zero memory and CPU overhead by running
QEMU. fake-machine doesn't get as close to zero as my original
fake QEMU target managed, but it is still pretty good, considering
how much less code is involved in fake-machine.

> diff --git a/cpu-exec.c b/cpu-exec.c
> index 8c9fb8b..cd1259a 100644
> --- a/cpu-exec.c
> +++ b/cpu-exec.c
> @@ -230,7 +230,7 @@ int cpu_exec(CPUState *env1)
>      uint8_t *tc_ptr;
>      unsigned long next_tb;
> 
> -    if (cpu_halted(env1) == EXCP_HALTED)
> +    if (fake_machine || cpu_halted(env1) == EXCP_HALTED)
> 
>          return EXCP_HALTED;


Daniel