Re: [PATCH V4 02/19] physmem: fd-based shared memory

[Peter Xu]

On Thu, Dec 12, 2024 at 03:38:00PM -0500, Steven Sistare wrote:
> On 12/9/2024 2:42 PM, Peter Xu wrote:
> > On Mon, Dec 02, 2024 at 05:19:54AM -0800, Steve Sistare wrote:
> > > @@ -2089,13 +2154,23 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t 
> > > size, ram_addr_t max_size,
> > >       new_block->page_size = qemu_real_host_page_size();
> > >       new_block->host = host;
> > >       new_block->flags = ram_flags;
> > > +
> > > +    if (!host && !xen_enabled()) {
> > 
> > Adding one more xen check is unnecessary.  This patch needed it could mean
> > that the patch can be refactored.. because we have xen checks in both
> > ram_block_add() and also in the fd allocation path.
> > 
> > At the meantime, see:
> > 
> > qemu_ram_alloc_from_fd():
> >      if (kvm_enabled() && !kvm_has_sync_mmu()) {
> >          error_setg(errp,
> >                     "host lacks kvm mmu notifiers, -mem-path unsupported");
> >          return NULL;
> >      }
> > 
> > I don't think any decent kernel could hit this, but that could be another
> > sign that this patch duplicated some file allocations.
> > 
> > > +        if ((new_block->flags & RAM_SHARED) &&
> > > +            !qemu_ram_alloc_shared(new_block, &local_err)) {
> > > +            goto err;
> > > +        }
> > > +    }
> > > +
> > >       ram_block_add(new_block, &local_err);
> > > -    if (local_err) {
> > > -        g_free(new_block);
> > > -        error_propagate(errp, local_err);
> > > -        return NULL;
> > > +    if (!local_err) {
> > > +        return new_block;
> > >       }
> > > -    return new_block;
> > > +
> > > +err:
> > > +    g_free(new_block);
> > > +    error_propagate(errp, local_err);
> > > +    return NULL;
> > >   }
> > 
> > IIUC we only need to conditionally convert an anon-allocation into an
> > fd-allocation, and then we don't need to mostly duplicate
> > qemu_ram_alloc_from_fd(), instead we reuse it.
> > 
> > I do have a few other comments elsewhere, but when I was trying to comment.
> > E.g., we either shouldn't need to bother caching qemu_memfd_check()
> > results, or do it in qemu_memfd_check() directly.. and some more.
> 
> Someone thought it a good idea to cache the result of qemu_memfd_alloc_check,
> and qemu_memfd_check will be called more often.  I'll cache the result inside
> qemu_memfd_check for the special case of flags=0.

OK.

> 
> > Then I think it's easier I provide a patch, and also show that it can be
> > also smaller changes to do the same thing, with everything fixed up
> > (e.g. addressing above mmu notifier missing issue).  What do you think as
> > below?
> 
> The key change you make is calling qemu_ram_alloc_from_fd instead of 
> file_ram_alloc,
> which buys the xen and kvm checks for free.  Sounds good, I will do that in 
> the
> context of my patch.
> 
> Here are some other changes in your patch, and my responses:
> 
> I will drop the "Retrying using MAP_ANON|MAP_SHARED" message, as you did.
> 
> However, I am keeping QEMU_VMALLOC_ALIGN, qemu_set_cloexec, and 
> trace_qemu_ram_alloc_shared.

I guess no huge deal on these, however since we're talking..  Is that
QEMU_VMALLOC_ALIGN from qemu_anon_ram_alloc()?

A quick dig tells me that it was used to be for anon THPs..

    commit 36b586284e678da28df3af9fd0907d2b16f9311c
    Author: Avi Kivity <avi@redhat.com>
    Date:   Mon Sep 5 11:07:05 2011 +0300

    qemu_vmalloc: align properly for transparent hugepages and KVM

And I'm guessing if at that time was also majorly for guest ram.

Considering that this path won't make an effect until the new aux mem
option is on, I'd think it better to stick without anything special like
QEMU_VMALLOC_ALIGN, until it's justified to be worthwhile.  E.g., Avi used
to explicitly mention this in that commit message:

    Adjust qemu_vmalloc() to honor that requirement.  Ignore it for small 
regions
    to avoid fragmentation.

And this is exactly mostly small regions when it's AUX.. probably except
VGA, but it'll be SHARED on top of shmem not PRIVATE on anon anyway... so
it'll be totally different things.

So I won't worry on that 2M alignment, and I will try to not carry over
that, because then trying to remove it will be harder.. even when we want.

For the 2nd.. Any quick answer on why explicit qemu_set_cloexec() needed?

For 3rd, tracepoint would definitely be fine whenever you feel necessary.

> 
> Also, when qemu_memfd_create + qemu_ram_alloc_from_fd fails, qemu should fail 
> and exit,
> and not fall back, because something unexpected went wrong.  David said the 
> same.

Why?  I was trying to rely on such fallback to make it work on e.g. Xen.
In that case, Xen fails there and fallback to xen_ram_alloc() inside the
later call to ram_block_add(), no?

> Thus we still need to pass errp to qemu_memfd_create().

-- 
Peter Xu