Re: [Qemu-devel] Secondary VGA passthrough not working, hitting linux kernel's pci_fixup_video

[Sander Eikelenboom]

Wednesday, January 15, 2014, 11:25:20 PM, you wrote:

> On Wed, Jan 15, 2014 at 03:10:35PM -0700, Bjorn Helgaas wrote:
>> [+cc Jesse, David]
>> 
>> On Wed, Jan 15, 2014 at 12:19 PM, Sander Eikelenboom
>> <address@hidden> wrote:
>> 
>> >>> >>         I understood there is no bridge for a VGA device in your 
>> >>> >> virtual machine.
>> >>> >>         Your emulator for the bridge control register is odd.
>> >>>
>> >>> > That seems beside the point as there's no bridge here.
>> >>>
>> >>> >>         I guess your virtual machine ignore "PCI-to-PCI Bridge 
>> >>> >> Architecture
>> >>> >>         Specification".
>> >>>
>> >>> > Since there's no pci to pci bridge in this VM, why would this
>> >>> > specification apply?
>> 
>> My guess is that he is referring to sec 11.3.2 "Initialization
>> Algorithm" of the Bridge spec r1.2.  That section (and chapter 24 of
>> MindShare PCI System Architecture, 4th edition) apparently reiterate a
>> discovery algorithm that appears in the PCI spec.  I can't find that
>> algorithm in the 3.0 or 2.3 PCI specs, but I assume it was in an
>> earlier version.

> It's not in 2.2 either apparently.

>> > Yes i have done some more reading .. from what i could find from PCI specs 
>> > there is nothing in there
>> > that points to a specific boot vga card.
>> > The whole agp/pci/pcie bios selection is ACPI stuff .. but if all the 
>> > cards are of the same type ..
>> > there is no way to specify a specific card.
>> 
>> The algorithm from sec 11.3.2 *does* seem to give a way to identify a
>> specific boot VGA card, i.e., the first one you find when searching in
>> the specified order.  There are still issues, such as the fact that
>> 11.3.2 says to start at PCI bus 0, while MindShare says to start with
>> the largest bus number.

> The spec merely says walk topology top to bottom.
> If there are no bridges there's a single bus and it
> does not seem to apply.

>> Also, sec 12.1.2.1 of the Bridge spec says the PCI_COMMAND memory and
>> I/O enables do apply to VGA accesses, so that is potentially a way to
>> make two sibling VGA devices work, e.g., by toggling those bits to
>> control which device you want to talk to.  I also assume that when
>> identifying the "boot VGA device," one might ignore devices that the
>> BIOS left disabled.

> Absolutely. But the disabled device can't be used at all.
> Sander here wants to use one device in VGA mode, and the
> second device in non VGA mode.

> This can only work if the non VGA one is behind bridge, or
> if second device doesn't use the legacy VGA addresses.

>> >>> >> 00:03.0 VGA compatible controller [0300]: Cirrus Logic GD 5446 
>> >>> >> [1013:00b8] (prog-if 00 [VGA controller])
>> >>> >>         Subsystem: Red Hat, Inc Device [1af4:1100]
>> >>> >>         Physical Slot: 3
>> >>> >>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- 
>> >>> >> ParErr- Stepping- SERR- FastB2B- DisINTx-
>> >>> >>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast 
>> >>> >> >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>> >>> >>         Latency: 0
>> >>> >>         Region 0: Memory at f0000000 (32-bit, prefetchable) [size=32M]
>> >>> >>         Region 1: Memory at f30b0000 (32-bit, non-prefetchable) 
>> >>> >> [size=4K]
>> >>> >>         Expansion ROM at f30a0000 [disabled] [size=64K]
>> >>> >>
>> >>> >> 00:05.0 VGA compatible controller [0300]: Advanced Micro Devices 
>> >>> >> [AMD] nee ATI Turks [Radeon HD 6570] [1002:6759] (prog-if 00 [VGA 
>> >>> >> controller])
>> >>> >>         Subsystem: PC Partner Limited Device [174b:e193]
>> >>> >>         Physical Slot: 5
>> >>> >>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- 
>> >>> >> ParErr- Stepping- SERR- FastB2B- DisINTx+
>> >>> >>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast 
>> >>> >> >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>> >>> >>         Latency: 0
>> >>> >>         Interrupt: pin A routed to IRQ 81
>> >>> >>         Region 0: Memory at e0000000 (64-bit, prefetchable) 
>> >>> >> [size=256M]
>> >>> >>         Region 2: Memory at f3060000 (64-bit, non-prefetchable) 
>> >>> >> [size=128K]
>> >>> >>         Region 4: I/O ports at c100 [size=256]
>> >>> >>         Expansion ROM at f3080000 [disabled] [size=128K]
>> 
>> I am not an expert on this legacy stuff, but this looks like an
>> illegal configuration to me: both 00:03.0 and 00:05.0 claim to be VGA
>> controllers and both have I/O+ and Mem+, so I would think both would
>> respond to the legacy VGA addresses.

> Also both are on the same bus so no bridge filters these.


For completeness sake i have attached the dmesg from the guest booted with my 
patch and some additional printk's on top of that.
of interest i guess:

[    2.486917] vgaarb: device added: 
PCI:0000:00:03.0,decodes=io+mem,owns=io+mem,locks=none
[    2.490170] vgaarb: device added: 
PCI:0000:00:05.0,decodes=io+mem,owns=io+mem,locks=none
[    2.496679] vgaarb: loaded
[    2.496683] vgaarb: bridge control possible 0000:00:05.0
[    2.500011] vgaarb: no bridge control possible 0000:00:03.0

and after that the applying of the pci_video_fixup ..

[    2.918390] pci 0000:00:00.0: calling quirk_natoma+0x0/0x40
[    2.918394] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    2.933934] pci 0000:00:00.0: calling quirk_passive_release+0x0/0x90
[    2.934024] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    2.953087] pci 0000:00:01.0: calling quirk_isa_dma_hangs+0x0/0x40
[    2.953095] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    2.969218] pci 0000:00:01.2: calling quirk_usb_early_handoff+0x0/0x6c0
[    2.974714] xen: --> pirq=21 -> irq=23 (gsi=23)
[    2.983163] pci 0000:00:03.0: calling pci_fixup_video+0x0/0x200
[    2.983171] pci 0000:00:03.0: Current VGA default device: 03.0
[    2.983174] pci 0000:00:03.0: walked 1 bridges/busses for this device
[    2.983251] pci 0000:00:03.0: Boot video device
[    2.983329] pci 0000:00:05.0: calling pci_fixup_video+0x0/0x200
[    2.983330] pci 0000:00:05.0: Current VGA default device: 03.0
[    2.983333] pci 0000:00:05.0: walked 1 bridges/busses for this device

The "Current VGA default device: 03.0" line is a added printk at the start of 
pci_video_fixup ..
as you see vga_default_device is already set before the pci_video_fixup get's 
run. From what it seems
either from arch or vga arbitration code. Since this it *with* my patch .. it 
only keeps the vga_default_devive as
the Boot video device .. and only sets the shadow flag for that device.

dmesg.txt
Description: Text document