Re: [Qemu-devel] ARM QEMU/KVM and TrustZone

[Peter Maydell]

On 16 June 2012 18:37, Christoffer Dall <address@hidden> wrote:
>> On 22 May 2012 13:22, Peter Maydell <address@hidden> wrote:
>>> Historically for QEMU we haven't implemented TrustZone support even
>>> though we claim to emulate CPUs that provide it. Instead we provide a
>>> CPU which mostly looks like a variant of the real thing without the
>>> TrustZone feature. We then bolt on a few extra cp15 registers (eg the
>>> SCR) as a pragmatic move to get Linux guests to run. Now we're also
>>> dealing with KVM on ARM I'd like to define things a bit more solidly
>>> so KVM and TCG agree on what the CPU model they present is.
>>>
>>> There are several possible environments we could provide
>>> to a guest:
>>>  (1) a CPU with full TrustZone support
>>>  (2) a CPU without TrustZone at all
>>>  (3) a TZ CPU running in NonSecure PL0/PL1
>>>  (4) a TZ CPU running in Secure PL0/PL1
>>>
>>> In some ways (1) is the "purist" solution -- emulate exactly what the
>>> hardware does. However:
>>>
>>>  * on TCG it would require a lot of work, including new functionality
>>>   in core QEMU (to support having different CPU cores being able to
>>>   see different views of memory, and having the S/NS attribute
>>>   attached to memory transactions)
>>>
>>>  * it isn't possible in KVM, because the ARM Virtualization Extensions
>>>   don't allow you to fake the CPSR a guest sees, and so you can't
>>>   make the guest believe it is in Monitor mode
>>>
>>> Option (2) is architecturally sanctioned (ie TrustZone is an optional
>>> feature, not mandatory), but it doesn't correspond to real CPUs, in
>>> that the hardware Cortex-A8/A9/A15 always have TrustZone. So we're
>>> modelling something that doesn't really exist.
>>>
>>> Options (3) and (4) correspond to the environment an OS guest
>>> typically actually uses on hardware. For ARM's devboards (versatile
>>> express etc) Linux runs in the Secure world but it doesn't actually
>>> use any of the TrustZone functionality, it's just a "give me full
>>> access to everything" setup. For just about every other ARM system,
>>> the boot rom or equivalent keeps Secure world to itself, and the OS
>>> kernel runs in the NonSecure world. (This typically means that the
>>> boot rom provides a set of board-specific entry points via the Secure
>>> Monitor Call (SMC) instruction for doing operations like "invalidate
>>> whole L2 cache" which require secure privileges.)
>>>
>
> Is there anything preventing people from writing a small bootloader
> that switches into non-secure mode and runs kernels there as a general
> approach (apart from laziness)?

I'm not sure what you're suggesting here. Mostly kernels do run
in NS mode on hardware, except for on ARM devboards. On ARM
devboards if you try to run the kernel in NS mode it will fall
over the first time it tries something that needs secure world
privileges. For KVM my rule of thumb is that it needs to run
the same kernel the hw runs.

To some extent the small boot loader would be the thing I describe
as a 'fake bootrom' below.

>>> Proposal:
>>>
>>> My suggestion is that we present the guest with a view that looks like
>>> a sort of superset of (2) (3) and (4), ie sufficient that a guest
>>> expecting any of those environments can run. In particular:
>>>
>>>  * no cp15 registers have secure/nonsecure banking
>>>  * there is only one memory space visible
>>>  * secure-access-only permissions are not enforced
>>>  * the handful of only-in-trustzone registers are implemented
>>>   (eg VBAR, MVBAR)
>>>  * we implement a "fake monitor mode"
>>>
>>> The aim of the "fake monitor mode" is to allow us to provide fake
>>> qemu-specific bootroms which implement whatever the board's SMC
>>> interface is, without having to write specific KVM kernel code for
>>> each board. So we don't have to run arbitrary secure-world guest code.
>>> The rules are:
>>>  * on an SMC instruction we enter the guest at the SMC vector
>>>   as defined by the MVBAR (monitor vector base address register)
>>>  * we actually run with the same access permissions as above
>>>   (and under KVM if you look at CPSR.M it will tell you you're
>>>   in Supervisor mode)
>>>  * return from the SMC is via a standard exception return insn
>>>  * we don't implement the separate memory space for the secure
>>>   world. (This implies that you need to find space in the
>>>   non-secure world's physical memory map for the bootrom shim;
>>>   not a big deal I think since we already have a requirement
>>>   for some space to put QEMU's arm_boot trivial bootloader.)
>>>
>
> you could have a separate set of stage-2 translation tables for this
> and keep things separate for real (or would we rely on fake-SMC code
> to directly be able to read fake-non-secure data, which is still
> possible through a different memory map I guess).

I'm trying to keep things simple (and separate memory maps for TCG
is a bit tricky). We can't make the monitor mode really look like
monitor mode, so there's not much point trying to implement all
the difficult complicated bits when we control the code running in
this mode anyway.

> what kind of operations would be required from SMC calls in a KVM
> guest setting? I can see this in an embedded market, but are they not
> likely to even capture Hyp mode already and set things up as required?
> What I mean is, if KVM is currently targeting Calxeda-type setups will
> we ever run kernels that require SMC operations as guests?

Yes, I think we must, because the CPU hardware doesn't let kernels
do everything in NS mode. There are usually only a handful of
operations needed. I've given some A8 examples above; I'm afraid
I don't have time to check for the A15 equivalents (I have a plane
to catch later :-)).

> It feels a bit premature to implement all this.

Basically it works at the moment for the vexpress A15 guest
because it happens to be one of the special cases which runs
in secure mode, and our cp15 emulation can just include
enough rope to let it all work. However that is likely to
drift into the ill-defined area where we are running the
guest in something that's actually NS but the cp15 emulation
lets it appear to access some S mode only registers.

In any case, I already have this problem in TCG mode for the
OMAP3 emulation, and so I want to define how it should all
work for that. We don't need to implement the KVM side just
yet as long as we're confident that we could...

-- PMM