Re: [Qemu-devel] [Qemu-ppc] [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default

[Andrea Bolognani]

On Mon, 2016-11-21 at 13:12 +1100, Alexey Kardashevskiy wrote:
> > > >    1) switch to PCI Express on newer machine types, and
> > > >       expose some sort of capability through QMP so that
> > > >       libvirt can know about the switch
> > > > 
> > > > [...]
> > > > Option 1) would break horribly with existing libvirt
> > > > versions, and so would Option 2) if we default to using
> > > 
> > > How exactly 1) will break libvirt? Migrating from pseries-2.7 to
> > > pseries-2.8 does not work anyway, and machines are allowed to behave
> > > different from version to version, what distinct difference will using
> > > "pseries-pcie-X.Y" make?
> > 
> > Existing libvirt versions assume that pseries guests have
> 
> If libvirt is using just "pseries" (without a version), then having a
> virtual PCIe-PCI bridge (and "pci.0" always available by default) will do it.

Please don't. Any device that is included in the guest
by default and can't be turned off makes libvirt's life
significantly more difficult (see below).

> If libvirt is using a specific version of pseries, then it already knows
> that <=2.7 has pci.0 as a root, pcie.0 otherwise. libvirt has a knowledge
> what QEMU version has what, right?

It doesn't yet, that's the point :)

We *could* add such knowledge to libvirt[1], but *existing*
libvirt versions would still not know about it, which means
that upgrading QEMU withough upgrading libvirt will result
in failure to create new guests.

> In what scenario will an additional machine type help?

Because then libvirt could learn that

  pseries-x.y       <->  pci.0
  pseries-pcie-x.y  <->  pcie.0

the same way it already knows that

  pc-i440fx-x.y     <->  pci.0
  pc-q35-x.y        <->  pcie.0

and choosing between one or the other would be, I think,
much easier for the user as well.

> > a legacy PCI root bus, and will base their PCI address
> > allocation / PCI topology decisions on that fact: they
> > will, for example, use legacy PCI bridges.
> > 
> > So if you used a new QEMU binary with a libvirt version
> > that doesn't know about the change, new guests would end up
> > using the wrong controllers. Existing guests would not be
> > affected as they would stick with the older machine types,
> > of course.
> > 
> > > I believe after we introduced the very first
> > > pseries-pcie-X.Y, we will just stop adding new pseries-X.Y.
> > 
> > Isn't i440fx still being updated despite the fact that q35
> > exists? Granted, there are a lot more differences between
> > those two machine types than just the root bus type.
> 
> I do not know about i440<->q35 but in pseries the difference is going to be
> very simple.
> 
> For example, we did not change the machine type when we switched from
> default OHCI to XHCI, switching from PCI to PCIe does not sound like we
> need a whole new machine type for this either.

The change from OHCI to XHCI only affected the *default* USB
controller, which libvirt tries its best not to use anyway:
instead, it will prefer to use '-M ...,usb=off' along with
'-device ...' and set both the controller model and its PCI
address explicitly, partially to shield its users from such
changes in QEMU.

Let's not forget that libvirt is a management layer, and as
such it needs to have as much control as possible over the
virtual hardware presented to the guest, mostly for guest ABI
compatibility purposes. Defaulting to XHCI instead of OHCI,
same as pretty much all other defaults, is good for people who
run QEMU directly, but libvirt needs to be able to control
such knobs itself in order to effectively perform the task it
was designed for.

Moreover, we're talking about a more fundamental change here:
the PCI Root Bus is not just any random device, it's the one
fundation upon which the entire PCI hierarchy is built. Based
on whether the machine exposes a PCI Express Root Bus or a
legacy PCI Express Root Bus, libvirt will create very
different PCI hierarchies, eg. using several ioh3420 devices
instead of a single pci-bridge device.

I'm still not sure if that enough to warrant an entirely new
machine type, but it definitely has a more far-reaching impact
than simply flipping the default USB controller from OHCI to
XHCI.

> > Even if no newer pseries-x.y were to be added after
> > introducing pseries-pcie, you could still easily create
> > guests that use either root bus, so no loss in functionality.
> 
> I could do this with the existing pseries if the machine had a "root but
> type" property.

That was indeed one of my original proposals ;)

Just note, once again, that the default for this property
would have to be "off" or we would run into the same issues
described above.


[1] Even though, as I mentioned earlier in the thread,
    performing version checks on machine types is frowned
    upon as it turns into a minefield as soon as backports
    and downstream machine types enter the picture... It's
    much better to expose some flag through QMP for libvirt
    to introspect
-- 
Andrea Bolognani / Red Hat / Virtualization