Re: [Qemu-devel] [PATCH v2 0/7] APIC/IOAPIC cleanup

[Blue Swirl]

On Thu, Aug 19, 2010 at 9:51 PM, Anthony Liguori <address@hidden> wrote:
> On 08/19/2010 04:21 PM, Blue Swirl wrote:
>>>
>>> Should CPUs appear in the QEMU device tree?
>>>
>>
>> They have several properties that should be user visible.
>>
>
> Sure, but that's an argument for having some of the qdev features (like
> variant properties) be consumable outside of qdev.

Or rather, separate qdev from SysBus. I don't think either makes much sense.

>>> requirement to sit on a bus.  A UART16650A does not sit on bus.  It sits
>>> on
>>> a card and is wired to the ISA bus or is sometimes wired directly to pins
>>> on
>>> a CPU on a SoC.
>>>
>>
>> Or all buses should be unified, so all devices could be wired into any
>> board.
>>
>
> But that defeats the notion of having bus level interfaces.  You really want
> the virtio PCI layer to only use PCI functions and specifically to interact
> with PCI concepts that don't exist in other busses (like IO regions and
> config space).  However, you also want the ability to interact with a virtio
> device through a well defined interface that isn't PCI specific.

True, but if a device uses dedicated bus interfaces, it can't be used
on other buses. It might be possible to construct the devices so that
most of the code is bus agnostic and only a shim interfaces with the
bus of the day, which should be close to what you proposed in the
earlier serial example.

One way to model pure devices could be something like chip select
line, address bus (possibly truncated) and data bus (also possibly
narrowed).

Any way, I think the current model is pretty much fine as is.

> The only way to do this right now is having a bus with a single device.
>  I've been working on the serial device, and what this requires is having an
> ISASerialDevice which is-a ISADevice.  The ISASerialDevice has-a SerialBus
> and the SerialBus contain a single SerialDevice which is-a DeviceState.
>
> But the ISASerialDevice has to assume that the SerialBus contains only a
> single child because it needs to connect the GPIO out from the SerialDevice
> to the ISA irq that's assigned.
>
> It would be a whole lot simpler to break the "all devices sit on busses"
> assumption that we have today and simply have the ISASerialDevice has-a
> SerialDevice with no additional layers of bus indirection.
>
>> IIRC I tried also that approach when I worked on this patch set but
>> there were some problems. Maybe with header file conflicts, or
>> qemu_irq was not available to CPU code.
>>
>
> Okay, I'll queue it up for a future day.
>
>>> Because not all devices on the sysbus can be hot added so if you made the
>>> bus hotpluggable, it would basically defeat the point of even marking a
>>> bus
>>> as not supporting hot plug.
>>>
>>> IOW, the whole bus is either hot pluggable or not.  You cannot just say
>>> that
>>> device X can be hotplugged but nothing else.
>>>
>>
>> Perhaps the hotplugging system in QEMU needs some rethinking instead.
>> Many real devices are not hot pluggable.
>>
>
> That's probably fair.  I don't think hot plugging should exist at the qdev
> level.  Hot plugging is a very bus specific concept with very bus specific
> restrictions.  For instance, with PCI, you can only plug at slot
> granularities whereas today, we don't really model multi-function devices as
> anything more than a set of unrelated devices.  So there's really no way you
> could conceptually hot plug a multi-function virtio adapter although you can
> pretty trivially create one statically.

Maybe PCI could introduce an indirection: cards. They could contain
any number of qdev devices.

>>>>> I think the options are to allow non-bus devices (like the APIC) or
>>>>> make
>>>>> the
>>>>> APIC a special case that's part of the CPU emulation.
>>>>>
>>>>>
>>>>
>>>> No. There could also be a new hotpluggable bus type, CPUBus, one
>>>> instance between each CPU and APIC. Or SysBusWithHotPlug. But I don't
>>>> see how that would be different from SysBus.
>>>>
>>>>
>>>
>>> Neither approach maps well to real hardware.  An x86 CPU cannot exist
>>> without a local APIC and a local APIC cannot exist without an x86 CPU.
>>>  The
>>> two are fundamentally tied together.
>>>
>>
>> What about 486? Or 82489?
>>
>
> Don't confuse the local APIC with the PIC or the I/O APIC.
>
> The local APIC has always existed in the CPU core.  There is also an I/O
> APIC which exists outside of the CPU core.  The local APIC was introduced
> with SMP support.

No, 82489 was an external local APIC for 486DX. See for example Intel
Multiprocessor Specification, page 3-5.

> The PIC and IO APIC totally make sense to be modelled as qdev.
>
>>>  It's like modelling a TLB as a
>>> separate device.
>>>
>>
>> That has surely happened in ancient times.
>>
>
> Yes, but the programming model was different.
>
> Look at the PIC compared to the lapic.  The PIC is programmed via pio at a
> fixed location.  There is only one PIC and it interacts with the system just
> like all other devices.  IOW, there is no reference to CPUState.
>
> When you look at the local APIC (apic.c) however, you see that it's the only
> device in the tree that actually interacts with a CPUState.  The notion of
> cpu_get_current_apic() is a good example of the tricks needed to make this
> work.
>
> The local APIC is a cpu feature, not a device.

That is certainly one way how to handle it, but not the only one and
not the most flexible. In contrast, CPUBus could have other uses like
north/south bridges, caches etc. On Sparc, a CPUBus could fuse ASIs
and addresses and these could be decoded by an external device. That
could simplify target-sparc/op_helper.c while not messing physical
address space order too much.

But in my mind, what in this specific case binds local APIC tighter to
the CPU than to the devices is the CR register access. We probably
don't want a CRBus.