Re: [PATCH RFCv2 00/20] kvm/arm: Introduce a customizable aarch64 KVM host model

[Marc Zyngier]

On Fri, 20 Dec 2024 11:52:51 +0000,
Kashyap Chamarthy <kchamart@redhat.com> wrote:
> 
> On Thu, Dec 19, 2024 at 03:41:56PM +0000, Marc Zyngier wrote:
> > On Thu, 19 Dec 2024 15:07:25 +0000,
> > Kashyap Chamarthy <kchamart@redhat.com> wrote:
> > > 
> > > On Thu, Dec 19, 2024 at 12:26:29PM +0000, Marc Zyngier wrote:
> > > > On Thu, 19 Dec 2024 11:35:16 +0000,
> > > > Kashyap Chamarthy <kchamart@redhat.com> wrote:
> 
> [...]
> 
> > > > You can't rely on userspace for security, that'd be completely
> > > > ludicrous.
> > > 
> > > As Dan Berrangé points out, it's the bog-standard way QEMU deals with
> > > some of the CPU-related issues on x86 today.  See this "important CPU
> > > flags"[2] section in the QEMU docs.
> > 
> > I had a look, and we do things quite differently. For example, the
> > spec-ctrl equivalent in implemented in FW and in KVM, and is exposed
> > by default if the HW is vulnerable. Userspace could hide that the
> > mitigation is there, but that's the extent of the configurability.
> 
> Noted.  As Dan says, as long as QEMU can toggle the feature on/off, then
> that might be sufficient in the context of migratability.
> 
> [...]
> 
> > > To reply to your other question on this thread[3] about "which ABI?"  I
> > > think Dan is talking about the *guest* ABI: the virtual "chipset" that
> > > is exposed to a guest (e.g. PCI(e) topology, ACPI tables, CPU model,
> > > etc).  As I understand it, this "guest ABI" should remain predictable,
> > > regardless of:
> > > 
> > >   - whether you're updating KVM, QEMU, or the underlying physical
> > >     hardware itself; or
> > >   - if the guest is migrated, live or offline
> > > 
> > > (As you might know, QEMU's "machine types" concept allows to create a
> > > stable guest ABI.)
> > 
> > All of this is under control of QEMU, *except* for the "maximum" of
> > the architectural features exposed to the guest. All you can do is
> > *downgrade* from there, and only to a limited extent.
> > 
> > That, in turn has a direct impact on what you call the "CPU model",
> > which for the ARM architecture really doesn't exist. All we have is a
> > bag of discrete features, with intricate dependencies between them.
> 
> I see; thanks for this explanation.  Your last sentence above is the
> shortest summary of the CPU features situation on ARM I've ever read so
> far. 
> 
> So, I infer this from what you're saying (do correct if it's wrong):
> 
>   • Currently it is impractical (not feasible?) to pull together a
>     minimal-and-usable set of CPU features + their dependencies on ARM
>     to come up with a "CPU model" that can work across a reasonable set
>     of hardware.

It isn't quite that. It *is* technically possible, and KVM does give
you the tools you need for that. In practice, the diversity of the
ecosystem is so huge that you can only rely on some very basic stuff
unless the implementations are already very close. And that "small
details" such as the timer frequency are strictly identical.

> 
>   • If the above is true, then the ability to toggle CPU features on and
>     off might become even more important for QEMU — if it wants to be
>     able to support live migration across mixed set of hardware on ARM.  

Turning CPU features off is not always possible. Hiding them is
generally possible, with a number of exceptions. We try our best to
provide both, but it's... complicated.

[...]

> Related tangent on CPU feature discoverability on ARM:
> 
> Speaking of "Neoverse-N1", looking at a system that I have access to,
> the `lscpu` output does not say anything about who the integrator is; it
> only says:
> 
>     ...
>     Vendor ID:                ARM
>     Model name:             Neoverse-N1
>     ...
> 
> I realize, `lscpu` displays only whatever the kernel knows.  Nothing in
> `dmidecode` either.

The kernel does not know anything about the "Neoverse-N1" string. It
can match some MIDR_EL1 values for errata workaround purposes, but
doesn't gives two hoots about a human readable string.

Every other year, we get asked to add a full database of strings in
the kernel. The answer is a simple, polite, and final "no way". This
serves no purpose at all. lscpu does have that database, and that's
the right place to do it.

When it comes to integration, the firmware can optionally report some
information, which is the EL3 version of a commercial break (see the
SOC_ID stuff). This isn't wildly deployed, thankfully.

> Also, it looks like there's no equivalent of a "CPUID" instruction (I
> realize it is x86-specific) on ARM.  Although, I came across a Google
> Git repo that seems to implement a bespoke, "aarch64_cpuid".  From a
> what I see, it seems to fetch the "Main ID Register" (MIDR_EL1) - I
> don't know enough about it to understand its implications:
> 
>     https://github.com/google/cpu_features/blob/main/src/impl_aarch64_cpuid.c

MIDR_EL1 doesn't give you much, and you cannot assume anything about
the the feature set from it. Linux already allows you to inspect the
ID registers from userspace (by trapping, emulating, and sanitising
the result). That's the only reliable source of information.

>
> > That's why I don't see CPU models as a viable thing in terms of ABI.
> > They are an approximation of what you could have, but the ABI is
> > elsewhere.
> 
> Hmm, this is "significant new information" for me.  If CPU models can't
> be part of the guest ABI on ARM, then the whole "migratability across
> heterogenous hardware" on QEMU requires deeper thinking.

As I said all along, the only source of truth is the set of ID
registers. Nothing else. You can build a "model" on top of that, but
not the other way around.

Thanks,

        M.

-- 
Without deviation from the norm, progress is not possible.