Re: [Qemu-devel] Re: [PATCH] PCI: Bus number from the bridge, not the device

[Michael S. Tsirkin]

On Sun, Nov 21, 2010 at 06:01:11PM +0200, Gleb Natapov wrote:
> > This FW is given to guest by qemu. It only assigns bus numbers
> > because qemu told it to do so.
> Seabios is just a guest qemu ships. There are other FW for qemu.

We don't support them though, do we?

> Bochs
> bios, openfirmware, efi. All of them where developed outside of qemu
> project and all of them are usable without qemu. You can't consider them
> be part of qemu any more then Linux/Windows with virtio drivers.

You can also burn linuxbios onto your motherboard. If you do you
void your warranty.


> > 
> > > > 
> > > > > > And the spec says, e.g.:
> > > > > > 
> > > > > >       the memory mapped configuration base
> > > > > >     address (always corresponds to bus number 0) for the PCI 
> > > > > > Segment Group
> > > > > >     of the host bridge is provided by _CBA and the bus range 
> > > > > > covered by the
> > > > > >     base address is indicated by the corresponding bus range 
> > > > > > specified in
> > > > > >     _CRS.
> > > > > > 
> > > > > Don't see how it is relevant. And _CBA is defined only for PCI 
> > > > > Express. Lets
> > > > > solve the problem for PCI first and then move to PCI Express. Jumping 
> > > > > from one
> > > > > to another destruct us from main discussion.
> > > > 
> > > > I think this is what confuses us.  As long as you are using cf8/cfc 
> > > > there's no
> > > > concept of a domain really.
> > > > Thus:
> > > >         /address@hidden
> > > > 
> > > > is probably enough for BIOS boot because we'll need to make root bus 
> > > > numbers
> > > > unique for legacy guests/option ROMs.  But this is not a hardware 
> > > > requirement
> > > > and might become easier to ignore eith EFI.
> > > > 
> > > You do not need MMCONFIG to have multiple PCI domains. You can have one
> > > configured via standard cf8/cfc and another one on ef8/efc and one more
> > > at mmio fce00000 and you can address all of them:
> > > /address@hidden
> > > /address@hidden
> > > /address@hidden

Isn't the mmio one relocatable?

> > > 
> > > And each one of those PCI domains can have 256 subbridges.
> > 
> > Will common guests such as windows or linux be able to use them? This
> With proper drivers yes. There is HW with more then one PCI bus and I
> think qemu emulates some of it (PPC MAC for instance). 

MAC is probably just poking in a couple of predefined places.  That's
not enumeration.

> > seems to be outside the scope of the PCI Firmware specification, which
> > says that bus numbers must be unique.
> They must be unique per PCI segment group.

We've come full circle, didn't we? i am saying we should let users
specify PCI Segment group+bus as opposed to the io port, which they
don't use.

> > 
> > > > > > 
> > > > > > > > 
> > > > > > > > That should be enough for e.g. device_del. We do have the need 
> > > > > > > > to
> > > > > > > > describe the topology when we interface with firmware, e.g. to 
> > > > > > > > describe
> > > > > > > > the ACPI tables themselves to qemu (this is what Gleb's patches 
> > > > > > > > deal
> > > > > > > > with), but that's probably the only case.
> > > > > > > > 
> > > > > > > Describing HW topology is the only way to unambiguously describe 
> > > > > > > device to
> > > > > > > something or someone outside qemu and have persistent device 
> > > > > > > naming
> > > > > > > between different HW configuration.
> > > > > > 
> > > > > > Not really, since ACPI is a binary blob programmed by qemu.
> > > > > > 
> > > > > APCI is part of the guest, not qemu.
> > > > 
> > > > Yes it runs in the guest but it's generated by qemu. On real hardware,
> > > > it's supplied by the motherboard.
> > > > 
> > > It is not generated by qemu. Parts of it depend on HW and other part 
> > > depend
> > > on how BIOS configure HW. _BBN for instance is clearly defined to return
> > > address assigned bu the BIOS.
> > 
> > BIOS is supplied on the motherboard and in our case by qemu as well.
> You can replace MB bios by coreboot+seabios on some of them.
> Manufacturer don't want you to do it and make it hard to do, but
> otherwise this is just software, not some magic dust.

They support common hardware but not all features will automatically work.

> > There's no standard way for BIOS to assign bus number to the pci root,
> > so it does it in device-specific way. Why should a management tool
> > or a CLI user care about these? As far as they are concerned
> > we could use some PV scheme to find root devices and assign bus
> > numbers, and it would be exactly the same.
> > 
> Go write KVM userspace that does that. AFAIK there is project out there
> that tries to do that. No luck so far. Your world view is very x86/Linux
> centric. You need to broaden it a little bit. Next time you propose
> something ask yourself will it work with qemu-sparc, qemu-ppc, qemu-amd.

Your view is very qemu centric. Ask yourself whether what you propose
will work with libvirt. Better yet, ask libvirt developers.

> > > > > Just saying "not really" doesn't
> > > > > prove much. I still haven't seen any proposition from you that 
> > > > > actually
> > > > > solve the problem. No, "lets use guest naming" is not it. There is no
> > > > > such thing as "The Guest". 
> > > > > 
> > > > > --
> > > > >                       Gleb.
> > > > 
> > > > I am sorry if I didn't make this clear.  I think we should use the 
> > > > domain:bus
> > > > pair to name the root device. As these are unique and 
> > > > 
> > > You forgot to complete the sentence :) But you made it clear enough and
> > > it is incorrect. domain:bus pair not only not unique they do not exist
> > > in qemu at all
> > 
> > Sure they do. domain maps to mcfg address for express. bus is used for
> mcfg is optional as far as I can see. You can compile out MMCONFIG
> support on Linux.

This doesn't mean that you will be able to address all devices
on all systems if you do.

> > cf8/cfc addressing. They are assigned by BIOS but since BIOS
> > is supplied with hardware the point is moot.
> Most PC hardware is supplied with Windows, so what? BIOS is a code that
> runs in a guest. It is part of a guest. Every line of code executed by
> vcpu belongs to a guest. No need to redefine things to prove you point.

Call it what you want. I care about the payload that the user supplies.
BIOS and ACPI are implementation details of qemu needed to run said
payload smoothly. Said payload might not know or care about
IO address used to program host pci bridge.

> > 
> > > and as such can't be used to address device. They are
> > > product of HW enumeration done by a guest OS just like eth0 or C:.
> > > 
> > > --
> > >                   Gleb.
> > 
> > There's a huge difference between BIOS and guest OS,
> Not true.
> 
> >                                            and between bus
> > numbers of pci root and of nested bridges.
> Really? What is it?

Look at the spec citations I provided. bus number for root must be set by
firmware (or default to 0). The rest can be done by the OS.

> > 
> > Describing hardware io ports makes sense if you are trying to
> > communicate data from qemu to the BIOS.  But the rest of the world might
> > not care.
> > 
> The part of the world that manage HW cares.

You really want libvirt to deal with ioports?  Why only that?
Let them emulate PCI while we are at it?

> You may need to add device
> from monitor before first line of BIOS is event executed. How can you
> rely on BIOS enumerate of devices in this case?

We can control the bus numbers that the BIOS will assign to pci root.
It's probably required to make them stable anyway.

> 
> --
>                       Gleb.