Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use

[Dr. David Alan Gilbert]

* Alexey Kardashevskiy (address@hidden) wrote:
> On 07/09/17 19:51, Dr. David Alan Gilbert wrote:
> > * Alexey Kardashevskiy (address@hidden) wrote:
> >> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593
> >>
> >> What happens ithere is that every virtio block device creates 2 address
> >> spaces - for modern config space (called "virtio-pci-cfg-as") and
> >> for busmaster (common pci thing, called after the device name,
> >> in my case "virtio-blk-pci").
> >>
> >> Each address_space_init() updates topology for every address space.
> >> Every topology update (address_space_update_topology()) creates a new
> >> dispatch tree - AddressSpaceDispatch with nodes (1KB) and
> >> sections (48KB) and destroys the old one.
> >>
> >> However the dispatch destructor is postponed via RCU which does not
> >> get a chance to execute until the machine is initialized but before
> >> we get there, memory is not returned to the pool, and this is a lot
> >> of memory which grows n^2.
> >>
> >> These patches are trying to address the memory use and boot time
> >> issues but tbh only the first one provides visible outcome.
> > 
> > Do you have a feel for how much memory is saved?
> 
> 
> The 1/4 saves ~33GB (~44GB -> 11GB) for a 2GB guest and 400 virtio-pci
> devices. These GB figures are the peak values (but it does not matter for
> OOM killer), memory gets released in one go when RCU kicks in, it just
> happens too late.

Nice saving!  Still, why is it using 11GB?
What's it like for more sane configurations, say 2-3 virtio devices - is
there anything noticable or is it just the huge setups?

Dave


> The 3/4 saves less, I'd say 50KB per VCPU (more if you count peaks but so
> much). Strangely, I do not see the difference in valgrind output when I run
> a guest with 1024 or just 8 CPUs, probably "massif" is not the right tool
> to catch this.
> 
> > 
> > Dave
> > 
> >> There are still things to polish and double check the use of RCU,
> >> I'd like to get any feedback before proceeding - is this going
> >> the right way or way too ugly?
> >>
> >>
> >> This is based on sha1
> >> 1ab5eb4efb Peter Maydell "Update version for v2.10.0 release".
> >>
> >> Please comment. Thanks.
> >>
> >>
> >>
> >> Alexey Kardashevskiy (4):
> >>   memory: Postpone flatview and dispatch tree building till all devices
> >>     are added
> >>   memory: Prepare for shared flat views
> >>   memory: Share flat views and dispatch trees between address spaces
> >>   memory: Add flat views to HMP "info mtree"
> >>
> >>  include/exec/memory-internal.h |   6 +-
> >>  include/exec/memory.h          |  93 +++++++++----
> >>  exec.c                         | 242 +++++++++++++++++++--------------
> >>  hw/alpha/typhoon.c             |   2 +-
> >>  hw/dma/rc4030.c                |   4 +-
> >>  hw/i386/amd_iommu.c            |   2 +-
> >>  hw/i386/intel_iommu.c          |   9 +-
> >>  hw/intc/openpic_kvm.c          |   2 +-
> >>  hw/pci-host/apb.c              |   2 +-
> >>  hw/pci/pci.c                   |   3 +-
> >>  hw/ppc/spapr_iommu.c           |   4 +-
> >>  hw/s390x/s390-pci-bus.c        |   2 +-
> >>  hw/vfio/common.c               |   6 +-
> >>  hw/virtio/vhost.c              |   6 +-
> >>  memory.c                       | 299 
> >> +++++++++++++++++++++++++++--------------
> >>  monitor.c                      |   3 +-
> >>  vl.c                           |   4 +
> >>  hmp-commands-info.hx           |   7 +-
> >>  18 files changed, 448 insertions(+), 248 deletions(-)
> >>
> >> -- 
> >> 2.11.0
> >>
> >>
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK
> > 
> 
> 
> -- 
> Alexey
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK