Re: Handling large allocations (bypassing mm?)

[Julian Andres Klode]

On Wed, Dec 14, 2022 at 04:11:18PM +0100, Daniel Kiper wrote:
> Adding a few folks who may be interested in this discussion too...
> 
> On Wed, Dec 14, 2022 at 02:21:49PM +0100, Julian Andres Klode wrote:
> > Hi,
> >
> > so I want to bring this discussion here that I had mostly with myself
> > in the past days on IRC.
> >
> > As some of you know, we had a couple issues with large initrds in
> > Ubuntu, Jeremy posted a patch series earlier about mmunlimited.
> 
> I skimmed through it but at first sight I do not like it. I will comment
> the patches more extensively in the following days.
> 
> > I wanted to propose a more fine-grained approach, as well as a
> > more generic approach to handling large allocations.
> >
> > The first issue one experiences when opening large initrds is
> > that grub_file_open() calls grub_verifier_open() which simply
> > grub_malloc()s a buffer for the size of the file.
> >
> > Later, for initrd, we have to allocate it a second time, in
> > the upstream tree that happens via relocator, in the rhboot
> > tree it allocates directly from EFI.
> >
> > Now my basic proposal is quite simple: We make grub_malloc()
> > and that relocator allocation code bypass the grub memory
> > management altogether and just do raw EFI page allocations
> > (provide two function pointers grub_mm_allocate_pages and
> > grub_mm_free_pages, and just call them if allocation size
> > is large[1]). e.g. at the start of grub_malloc:
> 
> In general I think we should stop using legacy Linux loader on UEFI
> platforms and migrate to LoadFile2 protocol. Relevant patches for ARM64
> are in master now. We are waiting for patches from Atish which will
> merge ARM64 implementation into RISC-V (you can find earlier version
> of the patch set here [1]). Then it could be extended into other
> architectures including AMD64. Of course this will not solve problem
> with extra allocation in verifiers. However, at least we should not
> have problems with large initrds and we will not deal with relocator
> on EFI platforms.
> 
> WRT verifiers extra allocation, as you said, this is a problem but it can
> be solved later. Now at least we should be sure it is possible to load
> big initrds with LoadFile2 protocol and there are no obstacles in the
> GRUB code for this thing.

To my knowledge, that is incorrect and LoadFile2 suppoort does nothing
to address the issue of large initrds:

- LoadFile2 support for the kernel has only been added very recently
  (2020) and people need to support 10 year old kernels (2012).

  How large initrds do they need? I don't know. Do I want them
  stranded? Not necessarily.

- We do not need to read the initrd and keep it in memory ahead of
  time presumably but at the moment we reuse the allocation paths
  and just copy them over afaik.

I do not know how LoadFile2 is implemented on the kernel side, but
it may reinforce the very issue because any allocations on the grub
side can't be reused, and grub may be unable to reuse the initrd
allocation by the verifier for the loader by relocator, so you
end up needing to allocate 3*initrd (2 in grub, and 1 in the
kernel).

By directly allocating pages for the verifier and releasing
them after it's done we can effectively go down to a memory
usage of 2*initrd size, and *both* can be allocated > 4GB,
so they can be large.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en