Re: [PATCH v5 00/79] refactor main RAM allocation to use hostmem backend

[Igor Mammedov]

On Mon, 17 Feb 2020 12:33:33 -0500
Igor Mammedov <address@hidden> wrote:

Paolo,

Can you merge this via your tree if possible, pls?
(You said that's about the time you'd be preparing pull req)

> v5:
>   - pick up new Reviewed-by-s
>   - drop "ppc/prep: use memdev for RAM" since machine was removed
>   - add patch "mips/mips_jazz: add max ram size check"
>   - extend comment for "default_ram_id" to clarify that's setting it serves
>     as optin flag for -m support. [03/79]
>   - rebase "arm/raspi: use memdev for RAM" once more
> 
> v4:
>   - pick up new Reviewed-by-s
>   - Fix access to uninitialized pagesize/hpsize in
>     "[PATCH REPOST v3 74/80] exec: cleanup  
> qemu_minrampagesize()/qemu_maxrampagesize()"
>   - Make explicitly provided memory-backend work by using string property 
> instead
>     of link so it would be possible to delay access to the backend to the time
>     when backends are initialized.
>   - added new patches to make explicit backend work nice with -m and do sanity
>     check on ram_size
>       'vl.c: move -m parsing after memory backends has been processed'
>       'vl.c: ensure that ram_size matches size of machine.memory-backend"
>     all this ram_size business needs cleanup too, but that's out of the scope
>     of this series.
>   - include m68k/q800 board into conversion
>   - drop patches that were merged through other trees
> 
> v3:
>   - due to libvirt not being ready, postpone till 5.1
>      * [PATCH v2 82/86] numa: forbid '-numa node,  mem' for 5.0 and newer 
> machine types
>      and depended
>        [PATCH v2 86/86] numa: remove deprecated implicit RAM distribution  
> between nodes
>   - drop as not related "[PATCH v2 85/86] numa: make exit() usage consistent"
>   - drop "[PATCH v2 76/86] post conversion default_ram_id cleanup"
>     so that default memory-backedend won't be created for boards that do not 
> care
>     about -m. Which makes -m optin feature. We should decide  what do in  case
>     board doesn't use -m (but that's out of scope of this series)
>   - use object_register_sugar_prop() instead of hacking compat props directly
>   - simplified/reworked aspeed patches
>   - s/RAM_ADDR_FMT/size_to_str()/
>   - rename 'ram-memdev' property to 'memory-backend'
>   - minor fixes to numa-test
>   - fixes for issues noticed during review of
>        [PATCH v2 66/86] ppc/{ppc440_bamboo,sam460x}: drop RAM size fixup
> 
> v2:
>   - fix compile errors on mingw32 host by introducing RAM_ADDR_UFMT [11/86]
>   - replace "[PATCH 43/86] hppa: drop RAM size fixup" with alternative
>     patches made by Philippe (which effectively do the same thing but other
>     way around)
>   - ppc440: fix crash and add suggested valid RAM size in error output.
>     s/ppc4xx_sdram_adjust/ppc4xx_sdram_prep/ and simplify it by removing
>     not necessary nested loop
>   - rebase on current master due to new conflicts
> 
> 
> Series removes ad hoc RAM allocation API 
> (memory_region_allocate_system_memory)
> and consolidates it around hostmem backend. It allows to
>  * resolve conflicts between global -mem-prealloc and hostmem's "policy" 
> option
>    fixing premature allocation before binding policy is applied
>  * simplify complicated memory allocation routines which had to deal with 2 
> ways
>    to allocate RAM.
>  * it allows to reuse hostmem backends of a choice for main RAM without adding
>    extra CLI options to duplicate hostmem features.
>    Recent case was -mem-shared, to enable vhost-user on targets that don't
>    support hostmem backends [1] (ex: s390)
>  * move RAM allocation from individual boards into generic machine code and
>    provide them with prepared MemoryRegion.
>  * clean up deprecated NUMA features which were tied to the old API (see 
> patches)
>     - "numa: remove deprecated -mem-path fallback to anonymous RAM"
>     - (POSTPONED, waiting on libvirt side) "forbid '-numa node,mem' for 5.0 
> and newer machine types"
>     - (POSTPONED) "numa: remove deprecated implicit RAM distribution between 
> nodes"
> 
> Conversion introduces a new machine.memory-backend property and wrapper code 
> that
> aliases global -mem-path and -mem-alloc into automatically created hostmem
> backend properties (provided memory-backend was not set explicitly given by 
> user).
> And then follows bulk of trivial patches that incrementally convert individual
> boards to using machine.memory-backend provided MemoryRegion.
> 
> Board conversion typically involves:
>  * providing MachineClass::default_ram_size and MachineClass::default_ram_id
>    so generic code could create default backend if user didn't explicitly 
> provide
>    memory-backend or -m options
>  * dropping memory_region_allocate_system_memory() call
>  * using convenience MachineState::ram MemoryRegion, which points to 
> MemoryRegion
>    allocated by ram-memdev
> On top of that for some boards:
>  * added missing ram_size checks (typically it were boards with fixed ram 
> size)
>  * ram_size fixups were replaced by checks and hard errors, forcing user to
>    provide correct "-m" values instead of ignoring it and continuing running.
> 
> After all boards are converted the old API is removed and memory allocation
> routines are cleaned up.
> 
> git tree for testing:
>   https://github.com/imammedo/qemu convert_main_ram_to_memdev_v5
> 
> previous rev:
>   https://github.com/imammedo/qemu convert_main_ram_to_memdev_v4
> 
> CC: Paolo Bonzini <address@hidden>
> 
[...]