[Qemu-devel] [PATCH v4 00/17] spapr: add support for pci hotplug

[Michael Roth]

These patches are based on ppc-next, and can also be obtained from:

https://github.com/mdroth/qemu/commits/spapr-hotplug-pci-v4

v4:
 * added documentation for sPAPR-based hotplug (Alexey)
 * reworked DR Connectors to be QOM devices, where sensor/indicator
   states are accessed via RTAS via object methods and also exposed
   via composition tree for introspection via qom-get/qom-fuse.
   attached devices are managed via state transitions handled by
   the DRC device (Alex)
 * DRC-related constants now defined in seperate header file,
   implemented as enum types where applicable
 * removed stub implementations of sensors that were not relevant
   to dynamic-reconfiguration. we now return "not implemented"
   if a guest attempts to access them via rtas-get-sensor or
   rtas-set-indicator-state
 * added DRC reset hooks to complete unplug for devices awaiting
   additional action from the guest before removal
 * incorporated endian fixes from Bharata and tested on ppc64le
   (Alex/Bharata)
 * used rtas_{ld,st} helpers in place of cpu_physical_memory_map
   for configure-connector implementation (Alex)
 * used b_* helper macros for properties related to OF PCI Binding
   (Alexey)
 * added dynamic-reconfiguration option to spapr-pci-host-bridge to
   enable/disable PCI hotplug for child bus
 * added pseries-2.3 machine and compat code to disable PCI hotplug by
   default for older machine types (Alex)
 * removed OF properties and DRC instances related to hotplugging of
   PHBs. this is not a prereq for PCI hotplug and will be handled as
   a separate series
 * moved generation of boot-time devices properties to common helper
   that can be re-used for memory, cpu, and phb. (Bharata)
 * re-organized patches so that pci, memory, cpu, phb should base
   cleanly on common set of patches implementing core DRC functionality
   (Bharata)
 * moved PCI 0-address fix to separate series (Alex)

v3:
 * dropped emulation of firmware-managed BAR allocation. this will be
   introduced via a follow-up series via a -machine flag and tied to
   a separate hotplug event to avoid a race condition with guest vs.
   "firmware"-managed BAR allocation, in conjunction with required
   fixes to rpaphp hotplug kernel module to utilize this mode.
 * moved drc_table into sPAPREnvironment (Alexey)
 * moved INDICATOR_* constants and friends into spapr_pci.c (Alexey)
 * use prefixes for global types (DrcEntry/ConfigureConnectorState) (Alexey)
 * updated for new hotplug interface (Alexey)
 * fixed get-power-level to report current power-level rather than
   desired (Alexey)
 * rebased to latest ppc-next

v2:
  * re-ordered patches to fix build bisectability (Alexey)
  * replaced g_warning with DPRINTF in RTAS calls for guest errors (Alexey)
  * replaced g_warning with fprintf for qemu errors (Alexey)
  * updated RTAS calls to use pre-existing error/success macros (Alexey)
  * replaced DR_*/SENSOR_* macros with INDICATOR_* for set-indicator/
    get-sensor-state (Alexey)

OVERVIEW

These patches add support for PCI hotplug for SPAPR guests. We advertise
each PHB as DR-capable (as defined by PAPR 13.5/13.6) with 32 hotpluggable
PCI slots per PHB, which models a standard PCI expansion device for Power
machines where the DRC name/loc-code/index for each slot are generated
based on bus/slot number.

This is compatible with existing guest kernel's via the rpaphp hotplug
module, and existing userspace tools such as drmgr/librtas/rtas_errd for
managing devices.

NOTES / ADDITIONAL DEPENDENCIES

This series relies on v1.2.19 or later of powerppc-utils (drmgr, rtas_errd,
ppc64-diag, and librtas components, specificially), which will automate
guest-side hotplug setup in response to an EPOW event emitted by QEMU. For
guests with older versions of powerpc-utils, a manual workaround must be
used (documented below).

Note that this relies on a patch to core PCI code which allows for the
use of a 0-address IO BAR for PCI devices. Without this patch, the first
hotplugged device will likely fail. This patch will be handled separately,
but is included in the in the development tree below for testing:

https://github.com/mdroth/qemu/commits/spapr-hotplug-pci

PATCH LAYOUT

Patches
        1     Documentation for sPAPR Dynamic-Reconfiguration/hotplug
        2     Initial implementation for sPAPRDRConnector device
        3-7   Guest RTAS calls to interact with DRC devices
        8-9   Introduce RTAS events for signalling hotplug operations
              to guest, using existing infrastructure of
              EPOW/check-exception events
        10    DRC helper code to populate DT descriptions of present DRC
              devices
        11    pseries-2.3 machine type to enable hotplug functionality by
              default, and leave disable for pre-2.2 to maintain migration
              compatibility.
        12    spapr-host-bridge option to selectively enable PCI hotplug/DR
              on a PHB-by-PHB basis
        13-17 PCI-specific hotplug hooks and DRC creation to enable PCI
              hotplug and hotplug events

USAGE

For guests with powerpc-utils 1.2.19+:
  hotplug:
    qemu:
      device_add e1000,id=slot0
  unplug:
    qemu:
      device_del slot0

For guests with powerpc-utils prior to 1.2.19:
  hotplug:
    qemu:
      device_add e1000,id=slot0
    guest:
      drmgr -c pci -s "C0" -n -a
      echo 1 >/sys/bus/pci/rescan
  unplug:
    guest:
      drmgr -c pci -s "C0" -n -r
      echo 1 >/sys/bus/pci/devices/0000:00:00.0/remove
    qemu:
      device_del slot0