[Qemu-devel] Plan for using softmmu with linux-user

[Richard Henderson]

[ Re-adding qemu-devel so that the plan can be critiqued. ]

On 08/13/2015 07:32 AM, Chen Gang wrote:
> On 8/13/15 22:05, Richard Henderson wrote:
>> On 08/13/2015 12:20 AM, gchen gchen wrote:
>>> Excuse me, I am not quite familiar with it, could any members provide
>>> more details about "use softmmu with linux-user" or "softmmu but without
>>> system emulation"?
>>
>> It doesn't exist yet.  It must be written.
>>
> 
> OK, thanks, and I shall try to implement it ...

My best guess that this is 3-4 months of work.

The current code within linux-user needs to be abstracted.  The new interface
must handle several things:

 (1) Finding unmapped portions of the guest address space.
 (2) Mapping anonymous memory into the guest address space.
 (3) Mapping files into the guest address space.
 (4) Filling in struct iovec for the host pages of a
     contiguous guest memory block.
 (5) Copying guest data to/from a contiguous host data block
     (like lock_user/unlock_user).

The current code has some of the above.

With the current implementation, (2) and (3) are mostly trivial.  There is
extra complex code to attempt these when host and guest page sizes differ, but
it's not completely correct.  And it definitely fails when 64-bit guests
running on 32-bit hosts attempt to map data above 4GB.

We don't currently have a need for (4), as contiguous guest pages currently
always imply contiguous host pages.  But with softmmu that will change, and the
speed of emulation for basic operations like read and write will be much
improved if we can issue readv and writev syscalls to the host os.

If (4) fails, or if there's no equivalent syscall that takes an iovec, then
copy in/out is the only solution.  Obviously we have a solution for the current
linux-user with lock_user/unlock_user.

The sets of patches that we should see will look something like this:

  (A1) Define the interface, and sets of callbacks.  This, IMO, should be done
       at the top level of qemu so that (eventually) both linux-user
       and bsd-user can share the code.

  (A2) Convert the existing two implementations to the set of callbacks.
       Note that I consider RESERVED_VA and !RESERVED_VA (unreserved_va?)
       different.  They do share some code, so callbacks for (4) and (5)
       will be the same, but certainly not the callbacks for (1), (2) or (3).

  (A3) Update linux-user to use the new interface, without (4).

  (A4) Update linux-user to use the new call for (4), for the syscalls
       that have iovec equivalents.

  (B) Replace the define CONFIG_SOFTMMU with a variable use_softmmu.
      In system mode (ifndef CONFIG_USER_ONLY), this should be a define that
      is always true, so that the dead code can be eliminated.  In user mode,
      this will be set depending on the guest mapping interface selected.

At this point you've really not changed anything, algorithmically. Everything
is working just the same as before you started, but the code is now in a form
where softmmu can be introduced.

  (C) Create the softmmu_user implementation.  You'll need something like the
      kernel's vm_area_struct to record ranges of guest address space
      mappings.  It'll need to be fast, as that data structure will provide
      the implementation for tlb_fill.

  (D) In the absence of command-line options, auto-select the most appropriate
      implementation:

      (1) If the host and guest page sizes don't match, select softmmu_user.
      (2) Otherwise, if 32-bit guest and 64-bit host, select reserved_va.
      (3) Otherwise, if a guest_base can be found to safely map the guest
          binary into the host address space (e.g. a 64-bit binary isn't
          linked above 4GB on a 32-bit host), select unreserved_va.
      (4) Otherwise, select softmmu.


r~