Re: [PATCH] spapr: Clarify error and documentation for broken KVM XICS

[Cédric Le Goater]

On 8/5/20 5:47 PM, Greg Kurz wrote:
> When starting an L2 KVM guest with `ic-mode=dual,kernel-irqchip=on`,
> QEMU fails with:
> 
> KVM is too old to support ic-mode=dual,kernel-irqchip=on
> 
> This error message was introduced to detect older KVM versions that
> didn't allow destruction and re-creation of the XICS KVM device that
> we do at reboot. But it is actually the same issue that we get with
> nested guests : when running under pseries, KVM currently provides
> a genuine XICS device (not the XICS-on-XIVE device that we get
> under powernv) which doesn't support destruction/re-creation.
> 
> This will eventually be fixed in KVM but in the meantime, update
> the error message and documentation to mention the nested case.
> While here, mention that in "No XIVE support in KVM" section that
> this can also happen with "guest OSes supporting XIVE" since
> we check this at init time before starting the guest.
> 
> Reported-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
> Buglink: https://bugs.launchpad.net/qemu/+bug/1890290
> Signed-off-by: Greg Kurz <groug@kaod.org>

Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C. 


> ---
>  docs/specs/ppc-spapr-xive.rst |    5 ++++-
>  hw/ppc/spapr_irq.c            |   12 +++++++++---
>  2 files changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst
> index 7199db730b82..7144347560f1 100644
> --- a/docs/specs/ppc-spapr-xive.rst
> +++ b/docs/specs/ppc-spapr-xive.rst
> @@ -126,6 +126,9 @@ xics            XICS KVM       XICS emul.     XICS KVM
>  
>  (1) QEMU warns with ``warning: kernel_irqchip requested but unavailable:
>      IRQ_XIVE capability must be present for KVM``
> +    In some cases (old host kernels or KVM nested guests), one may hit a
> +    QEMU/KVM incompatibility due to device destruction in reset. QEMU fails
> +    with ``KVM is incompatible with ic-mode=dual,kernel-irqchip=on``
>  (2) QEMU fails with ``kernel_irqchip requested but unavailable:
>      IRQ_XIVE capability must be present for KVM``
>  
> @@ -148,7 +151,7 @@ xics            XICS KVM       XICS emul.     XICS KVM
>      mode (XICS), either don't set the ic-mode machine property or try
>      ic-mode=xics or ic-mode=dual``
>  (4) QEMU/KVM incompatibility due to device destruction in reset. QEMU fails
> -    with ``KVM is too old to support ic-mode=dual,kernel-irqchip=on``
> +    with ``KVM is incompatible with ic-mode=dual,kernel-irqchip=on``
>  
>  
>  XIVE Device tree properties
> diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> index 2f8f7d62f875..72bb938375ef 100644
> --- a/hw/ppc/spapr_irq.c
> +++ b/hw/ppc/spapr_irq.c
> @@ -139,6 +139,7 @@ SpaprIrq spapr_irq_dual = {
>  
>  static int spapr_irq_check(SpaprMachineState *spapr, Error **errp)
>  {
> +    ERRP_GUARD();
>      MachineState *machine = MACHINE(spapr);
>  
>      /*
> @@ -179,14 +180,19 @@ static int spapr_irq_check(SpaprMachineState *spapr, 
> Error **errp)
>  
>      /*
>       * On a POWER9 host, some older KVM XICS devices cannot be destroyed and
> -     * re-created. Detect that early to avoid QEMU to exit later when the
> -     * guest reboots.
> +     * re-created. Same happens with KVM nested guests. Detect that early to
> +     * avoid QEMU to exit later when the guest reboots.
>       */
>      if (kvm_enabled() &&
>          spapr->irq == &spapr_irq_dual &&
>          kvm_kernel_irqchip_required() &&
>          xics_kvm_has_broken_disconnect(spapr)) {
> -        error_setg(errp, "KVM is too old to support 
> ic-mode=dual,kernel-irqchip=on");
> +        error_setg(errp,
> +            "KVM is incompatible with ic-mode=dual,kernel-irqchip=on");
> +        error_append_hint(errp,
> +            "This can happen with an old KVM or in a KVM nested guest.\n");
> +        error_append_hint(errp,
> +            "Try without kernel-irqchip or with kernel-irqchip=off.\n");
>          return -1;
>      }
>  
> 
>