Re: [Qemu-devel] Managing architectural restrictions with -device and libvirt

[Mark Cave-Ayland]

On 05/07/17 06:38, Markus Armbruster wrote:

> Copying Marcel for PCI expertise.
> 
> Mark Cave-Ayland <address@hidden> writes:
> 
>> Hi all,
>>
>> I've been working on a patchset that brings the sun4u machine on
>> qemu-system-sparc64 much closer to a real Ultra 5, however due to
>> various design restrictions I need to be able to restrict how devices
>> are added to the machine with -device.
>>
>> On a real Ultra 5, the root PCI bus (sabre) has 2 PCI bridges (simba A
>> and simba B) with the onboard devices attached to simba A with 2 free
>> slots, and an initially empty simba B.
>>
>> Firstly, is it possible to restrict the machine so that devices cannot
>> be directly plugged into the root PCI bus, but only behind one of the
>> PCI bridges? There is also an additional restriction in that slot 0
>> behind simba A must be left empty to ensure that the ebus (containing
>> the onboard devices) is the first device allocated.
> 
> I figure sabre, simba A, simba B and the onboard devices attached to
> simba A are all created by MachineClass init().

Yes that is effectively correct, although the Simba devices are created
as part of the PCI host bridge (apb) creation in pci_apb_init().

> What device provides "the ebus", and how is it created?

It's actually just an ISA bus, so the ebus device is effectively a
PCI-ISA bridge for legacy devices.

> Can you provide a list of all onboard PCI devices and how they are
> connected?  Diagram would be best.

I can try and come up with something more concise later, however I can
quickly give you the OpenBIOS DT from my WIP patchset if that helps:

0 > show-devs
ffe1bf38 /
ffe1c110 /aliases
ffe1c238 /openprom (BootROM)
ffe26b50 /openprom/client-services
ffe1c4f0 /options
ffe1c5d0 /chosen
ffe1c710 /builtin
ffe1c838 /builtin/console
ffe26618 /packages
ffe28640 /packages/cmdline
ffe28890 /packages/disk-label
ffe2c8d8 /packages/deblocker
ffe2cef0 /packages/grubfs-files
ffe2d300 /packages/sun-parts
ffe2d718 /packages/elf-loader
ffe2b210 /address@hidden,0 (memory)
ffe2b370 /virtual-memory
ffe2d878 /address@hidden,0 (pci)
ffe2e1a8 /address@hidden,0/address@hidden,1 (pci)
ffe2e960 /address@hidden,0/address@hidden,1/address@hidden
ffe2f1b0 /address@hidden,0/address@hidden,1/address@hidden/address@hidden
ffe2f328 /address@hidden,0/address@hidden,1/address@hidden/address@hidden 
(block)
ffe2f878 /address@hidden,0/address@hidden,1/address@hidden/address@hidden 
(serial)
ffe2fc08 /address@hidden,0/address@hidden,1/address@hidden/address@hidden (8042)
ffe2fe00 
/address@hidden,0/address@hidden,1/address@hidden/address@hidden/address@hidden 
(serial)
ffe301b0 /address@hidden,0/address@hidden,1/address@hidden,1 (network)
ffe307c8 /address@hidden,0/address@hidden,1/QEMU,address@hidden (display)
ffe31e40 /address@hidden,0/address@hidden,1/address@hidden (ide)
ffe32398 /address@hidden,0/address@hidden,1/address@hidden/address@hidden (ide)
ffe32678 /address@hidden,0/address@hidden,1/address@hidden/address@hidden (ide)
ffe32910 
/address@hidden,0/address@hidden,1/address@hidden/address@hidden/address@hidden 
(block)
ffe32f98 /address@hidden,0/address@hidden (pci)
ffe336e8 /SUNW,UltraSPARC-IIi (cpu)
 ok

For comparison you can see the DT from a real Ultra 5 here:
http://www.pearsonitcertification.com/articles/article.aspx?p=440286&seqNum=7

> The real sabre has two slots, and doesn't support hot (un)plug.  Can we
> simply model that?  If yes, the root PCI bus is full after init(), and
> remains full.  Takes care of "cannot directly plugged into the root PCI
> bus".

Right. So what you're saying is that if we add the 2 simba devices to
the sabre PCI host bridge during machine init and then mark the sabre
PCI root bus as not hotplug-able then that will prevent people adding
extra devices from the command line via -device? I will see if I can
find time to try this later this evening.

My understanding from reading various bits of documentation is that the
the empty simba bridge (bus B) can hold a maximum of 4 devices, whilst
the non-empty simba bridge (bus A) can hold a maximum of 2 devices
(presumably due to the on-board hardware). And in order to make sure
OpenBIOS maps the PCI IO ranges correctly, the ebus must be the first
on-board device found during a PCI bus scan which means slot 0 on bus A
must be blacklisted.

I guess what I'm looking for is some kind of hook that runs after both
machine init and all the devices have been specified on the command
line, which I can use to validate the configuration and provide a
suitable error message/hint if the configuration is invalid?


ATB,

Mark.