Re: [RFC 0/9] Add an interVM memory sharing device

[Igor Kotrasiński]

On 2/5/20 3:49 PM, Jan Kiszka wrote:
> On 05.02.20 15:39, Stefan Hajnoczi wrote:
> > On Tue, Feb 04, 2020 at 12:30:42PM +0100, 
> > address@hidden wrote:
> > > From: Igor Kotrasinski <address@hidden>
> > >
> > > This patchset adds a "memory exposing" device that allows two QEMU
> > > instances to share arbitrary memory regions. Unlike ivshmem, it does not
> > > create a new region of memory that's shared between VMs, but instead
> > > allows one VM to access any memory region of the other VM we choose to
> > > share.
> > >
> > > The motivation for this device is a sort of ARM Trustzone "emulation",
> > > where a rich system running on one machine (e.g. x86_64 linux) is able
> > > to perform SMCs to a trusted system running on another (e.g. OpTEE on
> > > ARM). With a device that allows sharing arbitrary memory between VMs,
> > > this can be achieved with minimal changes to the trusted system and its
> > > linux driver while allowing the rich system to run on a speedier x86
> > > emulator. I prepared additional patches for linux, OpTEE OS and OpTEE
> > > build system as a PoC that such emulation works and passes OpTEE tests;
> > > I'm not sure what would be the best way to share them.
> > >
> > > This patchset is my first foray into QEMU source code and I'm certain
> > > it's not yet ready to be merged in. I'm not sure whether memory sharing
> > > code has any race conditions or breaks rules of working with memory
> > > regions, or if having VMs communicate synchronously via chardevs is the
> > > right way to do it. I do believe the basic idea for sharing memory
> > > regions is sound and that it could be useful for inter-VM communication.
> >
> > Hi,
> > Without having looked into the patches yet, I'm already wondering if you
> > can use the existing -object
> > memory-backend-file,size=512M,mem-path=/my/shared/mem feature for your
> > use case?
> >
> > That's the existing mechanism for fully sharing guest RAM and if you
> > want to share all of memory then maybe a device is not necessary - just
> > share the memory.

That option adds memory in addition to the memory allocated with the 
'-m' flag, doesn't it? I looked into that option, and it seemed to me 
you can't back all memory this way.

Apart from that, the only advantage my solution has is that it's aware 
of any memory overlaying the memory-backed regions (i.e. memory backed 
by accessor functions). Maybe I don't need this for my use case, I'd 
have to test that.

> I suspect it's about sharing that memory in a discoverable way. Maybe it 
> is also about the signalling channel defined in the device.
>
> OTOH, when it's really about sharing everything, even bidirectional, 
> that rather looks like a pragmatic shortcut, not a generic model.
>
> The patches should clarify their use case a bit further, I think. The 
> title suggests a generic sharing solution, but my impression is that it 
> rather caters a specific case under specific boundary conditions.
>
> Jan

The solution does stem from a specific use case, the ARM Trustzone 
forwarding described in the cover letter. Normally both OSes can pass 
data around by sharing physical addresses (potentially from anywhere in 
memory), so giving VMs an ability to access one another's memory no 
matter how it's backed allows for minimal trusted OS modification, just 
offsetting physical addresses. The interrupt functionality also reflects 
this, it's intended to pass around SMC data.

I realize that this kind of total memory sharing couples the two VMs 
tightly - this is why I'm asking for comments on this, perhaps there's a 
better solution for this specific scenario.

For what it's worth, the extent of this sharing can be reduced by using 
a limited MemoryRegion like it's done for secure and non-secure memory 
views on ARM.

Igor