Re: [RFC 0/9] Add an interVM memory sharing device

[Igor Mammedov]

On Fri, 7 Feb 2020 10:00:50 +0100
Igor Kotrasiński <address@hidden> wrote:

> On 2/5/20 3:49 PM, Jan Kiszka wrote:
> > On 05.02.20 15:39, Stefan Hajnoczi wrote:  
> >> On Tue, Feb 04, 2020 at 12:30:42PM +0100, 
> >> address@hidden wrote:  
> >>> From: Igor Kotrasinski <address@hidden>
> >>>
> >>> This patchset adds a "memory exposing" device that allows two QEMU
> >>> instances to share arbitrary memory regions. Unlike ivshmem, it does not
> >>> create a new region of memory that's shared between VMs, but instead
> >>> allows one VM to access any memory region of the other VM we choose to
> >>> share.
> >>>
> >>> The motivation for this device is a sort of ARM Trustzone "emulation",
> >>> where a rich system running on one machine (e.g. x86_64 linux) is able
> >>> to perform SMCs to a trusted system running on another (e.g. OpTEE on
> >>> ARM). With a device that allows sharing arbitrary memory between VMs,
> >>> this can be achieved with minimal changes to the trusted system and its
> >>> linux driver while allowing the rich system to run on a speedier x86
> >>> emulator. I prepared additional patches for linux, OpTEE OS and OpTEE
> >>> build system as a PoC that such emulation works and passes OpTEE tests;
> >>> I'm not sure what would be the best way to share them.
> >>>
> >>> This patchset is my first foray into QEMU source code and I'm certain
> >>> it's not yet ready to be merged in. I'm not sure whether memory sharing
> >>> code has any race conditions or breaks rules of working with memory
> >>> regions, or if having VMs communicate synchronously via chardevs is the
> >>> right way to do it. I do believe the basic idea for sharing memory
> >>> regions is sound and that it could be useful for inter-VM communication.  
> >>
> >> Hi,
> >> Without having looked into the patches yet, I'm already wondering if you
> >> can use the existing -object
> >> memory-backend-file,size=512M,mem-path=/my/shared/mem feature for your
> >> use case?
> >>
> >> That's the existing mechanism for fully sharing guest RAM and if you
> >> want to share all of memory then maybe a device is not necessary - just
> >> share the memory.  
> 
> That option adds memory in addition to the memory allocated with the 
> '-m' flag, doesn't it? I looked into that option, and it seemed to me 
> you can't back all memory this way.
with current QEMU you play with memory sharing using numa workaround

-m 512 \
-object memory-backend-file,id=mem,size=512M,mem-path=/my/shared/mem 
feature,share=on \
-numa node,memdev=mem

also on the list there is series that allows to share main ram
without numa workaround, see
  "[PATCH v4 00/80] refactor main RAM allocation to use hostmem backend"

with it applied you can share main RAM with following CLI:

-object memory-backend-file,id=mem,size=512M,mem-path=/my/shared/mem 
feature,share=on \
-m 512 \
-M virt,memory-backend=mem

> Apart from that, the only advantage my solution has is that it's aware 
> of any memory overlaying the memory-backed regions (i.e. memory backed 
> by accessor functions). Maybe I don't need this for my use case, I'd 
> have to test that.
> 
> > 
> > I suspect it's about sharing that memory in a discoverable way. Maybe it 
> > is also about the signalling channel defined in the device.
> > 
> > OTOH, when it's really about sharing everything, even bidirectional, 
> > that rather looks like a pragmatic shortcut, not a generic model.
> > 
> > The patches should clarify their use case a bit further, I think. The 
> > title suggests a generic sharing solution, but my impression is that it 
> > rather caters a specific case under specific boundary conditions.
> > 
> > Jan
> >   
> 
> The solution does stem from a specific use case, the ARM Trustzone 
> forwarding described in the cover letter. Normally both OSes can pass 
> data around by sharing physical addresses (potentially from anywhere in 
> memory), so giving VMs an ability to access one another's memory no 
> matter how it's backed allows for minimal trusted OS modification, just 
> offsetting physical addresses. The interrupt functionality also reflects 
> this, it's intended to pass around SMC data.
> 
> I realize that this kind of total memory sharing couples the two VMs 
> tightly - this is why I'm asking for comments on this, perhaps there's a 
> better solution for this specific scenario.
> 
> For what it's worth, the extent of this sharing can be reduced by using 
> a limited MemoryRegion like it's done for secure and non-secure memory 
> views on ARM.
> 
> Igor
>