qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] Secondary VGA passthrough not working, hitting linux ke


From: Sander Eikelenboom
Subject: Re: [Qemu-devel] Secondary VGA passthrough not working, hitting linux kernel's pci_fixup_video
Date: Wed, 15 Jan 2014 14:14:49 +0100

Wednesday, January 15, 2014, 12:58:20 PM, you wrote:

> On Wed, Jan 15, 2014 at 11:37:29AM +0100, Sander Eikelenboom wrote:
>> Hi Michael / Anthony,
>> 
>> I'm trying to do secondary VGA passthrough with on Xen for some time. The 
>> problem was that the driver
>> would get the rom of the emulated VGA device instead of it's own rom from 
>> the rom bar.
>> 
>> In the end I found out it is because of a "fixup" in the linux kernel, 
>> "pci_fixup_video" in arch/x86/pci/fixup.c
>> that sets the IORESOURCE_ROM_SHADOW flag for both VGA cards and make both 
>> point to C000 (which is were the emulated rom is).
>> 
>> The code there tries to verify if the bridge has the PCI_BRIDGE_CTL_VGA bit 
>> set, and would only apply the fix when that's true
>> however the code doesn't get that far since it returns prematurely since 
>> !bridge (the code in question is below as is a dump of lspci -vvvknn).
>> 
>> The author of that code replied:
>> 
>>         I understood there is no bridge for a VGA device in your virtual 
>> machine.
>>         Your emulator for the bridge control register is odd.

> That seems beside the point as there's no bridge here.

>>         I guess your virtual machine ignore "PCI-to-PCI Bridge Architecture
>>         Specification".

> Since there's no pci to pci bridge in this VM, why would this
> specification apply?



>>         Thank you,
>> 
>>         Eiichiro Oiwa

> I can't see that reply anywhere except quoted in your mail.

Hmm strange his replies are indeed not archived in that LKML archive.


> Tried to hunt addresses for relevant people and Cc them all.
>         Cc: address@hidden
>         Cc: address@hidden
>         Cc: address@hidden
>         Cc: address@hidden
>         Cc: address@hidden
>         Cc: address@hidden

> not yet copying until we think it's a linux bug.

Thanks yes i thought it would be more wise to first verify with you Qemu guys 
:-)

Qemu has perhaps a slightly different pci setup/layout from the average real 
machine ..
However the linux kernel fixup seems to target quite broad for a very small 
number of cases it would actually fix.
So yes .. i don't know if there is actually any part to "blame" .. and if not 
.. where it would be the most easy
and durable to fix it.

For reference this is the linux history of commits that introduced and changed 
the current pci_fixup_video:

commit 6b5c76b8e2ff204fa8d7201acce461188873bf2b
Author: Eiichiro Oiwa <address@hidden>
Date:   Mon Oct 23 15:14:07 2006 +0900

    PCI: fix pci_fixup_video as it blows up on sparc64

    This reverts much of the original pci_fixup_video change and makes it
    work for all arches that need it.

    fixed, and tested on x86, x86_64 and IA64 dig.

    Signed-off-by: Eiichiro Oiwa <address@hidden>
    Acked-by: David Miller <address@hidden>
    Signed-off-by: Greg Kroah-Hartman <address@hidden>

commit 938f667198179dc0c8424e2cfac9cd9fe405bee3
Author: Paolo Ciarrocchi <address@hidden>
Date:   Wed Jan 30 13:33:00 2008 +0100

    x86: coding style fixes in arch/x86/pci/fixup.c
    Simple coding style fixes.
    no code changed:

commit 73e3b590f38fb7c03ee370430348edf1f401204e
Author: Yinghai Lu <address@hidden>
Date:   Thu Feb 23 23:46:52 2012 -0800

    PCI: Use class for quirk for pci_fixup_video

commit 6cf20beec4b91c240cf759b4db72669e251f1fc4
Author: Dave Airlie <address@hidden>
Date:   Mon May 14 17:00:40 2012 +0100

    x86/vga: set the default device from the fixup.

    Since Matthew's efi/vga changes on non-EFI machines we were failing
    to tell the vgaarb/switcheroo what the default device was, this
    sets the default device in the quirk if none has been set before.

    This fixes the switcheroo on my T410s.

commit a18e3690a52790a034d6540d54e8e1f1cd125da2
Author: Greg Kroah-Hartman <address@hidden>
Date:   Fri Dec 21 14:02:53 2012 -0800

    X86: drivers: remove __dev* attributes.

    CONFIG_HOTPLUG is going away as an option.  As a result, the __dev*
    markings need to be removed.

    This change removes the use of __devinit, __devexit_p, __devinitconst,
    and __devexit from these drivers.

    Based on patches originally written by Bill Pemberton, but redone by me
    in order to handle some of the coding style issues better, by hand.


>> 
>> So my first question is .. (i haven't got much knowledge on PCI) is he right 
>> and is the layout of pci devices in Qemu indeed out of spec ?
>> Or is the pci_fixup_video code assuming too much ... ?
>> 
>> And second .. any ideas of how to fix it ?
>> 
>> When i prevent this quirk/fixup from running in the guest kernel, the 
>> passthrough works fine .. it can read it's bios .. playing HD video .. even 
>> DPM seems to work with the radeon :-)
>> 
>> Also see this thread on the linux-kernel mailing list: 
>> https://lkml.org/lkml/2014/1/11/203


> Overall what's the issue here?

> Here's my analysis:
> You have two video cards both on the root bus.
> So the logic in fixup that tries to guess which one
> to use based on which one is behind bridge
> isn't effective.

Check.

> As the result fixup will enable ROM shadow for both,
> unsurprisingly this doesn't work.

Check.

> But they can't both work in VGA mode anyway, right?
> They would try to claim both addresses.

I don't know why .. but it seems they don't try to ..
probably how seabios works .. since it finds the optionrom for the emulated 
device and
enables VGA text mode at that moment ?

(d21) [2014-01-14 23:51:07] Invoking SeaBIOS ...
(d21) [2014-01-14 23:51:07] SeaBIOS (version 
rel-1.7.3-117-g31b8b4e-dirty-20140112_142355-serveerstertje)
(d21) [2014-01-14 23:51:07]
(d21) [2014-01-14 23:51:07] Found Xen hypervisor signature at 40000000
(d21) [2014-01-14 23:51:07] Running on QEMU (i440fx)
(d21) [2014-01-14 23:51:07] xen: copy e820...
(d21) [2014-01-14 23:51:07] Relocating init from 0x000e1b81 to 0x3f7e0e50 (size 
61675)
(d21) [2014-01-14 23:51:07] CPU Mhz=3202
(d21) [2014-01-14 23:51:07] Found 9 PCI devices (max PCI bus is 00)
(d21) [2014-01-14 23:51:07] Allocated Xen hypercall page at 3f7ff000
(d21) [2014-01-14 23:51:07] Detected Xen v4.4-unstable
(d21) [2014-01-14 23:51:07] xen: copy BIOS tables...
(d21) [2014-01-14 23:51:07] Copying SMBIOS entry point from 0x00010010 to 
0x000f0c50
(d21) [2014-01-14 23:51:07] Copying MPTABLE from 0xfc001190/fc0011a0 to 
0x000f0b40
(d21) [2014-01-14 23:51:07] Copying PIR from 0x00010030 to 0x000f0ac0
(d21) [2014-01-14 23:51:07] Copying ACPI RSDP from 0x000100b0 to 0x000f0a90
(d21) [2014-01-14 23:51:07] Using pmtimer, ioport 0xb008
(d21) [2014-01-14 23:51:07] Scan for VGA option rom
(d21) [2014-01-14 23:51:07] Running option rom at c000:0003
(XEN) [2014-01-14 23:51:07] stdvga.c:147:d21 entering stdvga and caching modes
(d21) [2014-01-14 23:51:07] pmm call arg1=0
(d21) [2014-01-14 23:51:07] Turning on vga text mode console



> So I would say you should just disable cirrus on qemu command line with
> the -no-graphics flag.
> Then it won't conflict.

yes you would end up with primary passthrough, which is also a goal, but 
secondary
passthrough seemed to be an easier primary goal.

I will see if i can get this working but as far as i know stuff didn't work on 
Xen without applying
additional patches to hvmloader and or qemu.

> If you don't want your card to work in VGA mode,
> just want it to work in non VGA mode, we run into a problem
> in that there's no standard way to disable VGA addresses in devices 
> themselves.
> If you want this one solution is to put your card behind a pci to pci bridge 
> in qemu.
> Our bridges don't support VGA so your card won't work in VGA mode then.

Yes it's not working in VGA mode now .. it only starts giving output on the 
monitor
when the radeon driver loads.

I don't know how this would work on a normal machine with say 2 graphic cards ..
would those always be behing a different bridge ?
If not the fixup code in the kernel isn't selective enough.

Isn't there another way a system can indicate which device is the primary (as 
you can select in most real world bioses
(agp/pci/pcie/igd) ?
If not then a boot option and manually specifying / overriding the boot vga 
card would probably be the only way to fix it i guess.


Thanks for your prompt reply and analysis !

--
Sander


>> --
>> 
>> Sander
>> 
>> 
>> 
>> /*
>>  * Fixup to mark boot BIOS video selected by BIOS before it changes
>>  *
>>  * From information provided by "Jon Smirl" <address@hidden>
>>  *
>>  * The standard boot ROM sequence for an x86 machine uses the BIOS
>>  * to select an initial video card for boot display. This boot video
>>  * card will have it's BIOS copied to C0000 in system RAM.
>>  * IORESOURCE_ROM_SHADOW is used to associate the boot video
>>  * card with this copy. On laptops this copy has to be used since
>>  * the main ROM may be compressed or combined with another image.
>>  * See pci_map_rom() for use of this flag. Before we mark the device
>>  * with IORESOURCE_ROM_SHADOW we have to check if this is or should become
>>  * the primary video card, since this quirk is ran for all video devices.
>>  */
>> 
>> static void pci_fixup_video(struct pci_dev *pdev)
>> {
>>         struct pci_dev *bridge;
>>         struct pci_bus *bus;
>>         u16 config;
>> 
>>         /* Is VGA routed to us? */
>>         bus = pdev->bus;
>>         while (bus) {
>>                 bridge = bus->self;
>> 
>>                 /*
>>                  * From information provided by
>>                  * "David Miller" <address@hidden>
>>                  * The bridge control register is valid for PCI header
>>                  * type BRIDGE, or CARDBUS. Host to PCI controllers use
>>                  * PCI header type NORMAL.
>>                  */
>>                 if (bridge
>>                     && ((bridge->hdr_type == PCI_HEADER_TYPE_BRIDGE)
>>                        || (bridge->hdr_type == PCI_HEADER_TYPE_CARDBUS))) {
>>                         pci_read_config_word(bridge, PCI_BRIDGE_CONTROL,
>>                                                 &config);
>>                         if (!(config & PCI_BRIDGE_CTL_VGA))
>>                                 return;
>>                 }
>>                 bus = bus->parent;
>>         }
>>         if (!vga_default_device() || pdev == vga_default_device()) {
>>                 pci_read_config_word(pdev, PCI_COMMAND, &config);
>>                 if (config & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY)) {
>>                         pdev->resource[PCI_ROM_RESOURCE].flags |= 
>> IORESOURCE_ROM_SHADOW;
>>                         dev_printk(KERN_DEBUG, &pdev->dev, "Boot video 
>> device\n");
>>                         vga_set_default_device(pdev);
>>                 }
>>         }
>> }
>> DECLARE_PCI_FIXUP_CLASS_FINAL(PCI_ANY_ID, PCI_ANY_ID,
>>                                 PCI_CLASS_DISPLAY_VGA, 8, pci_fixup_video);
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> # lspci -vvvknn
>> 00:00.0 Host bridge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] 
>> [8086:1237] (rev 02)
>>         Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
>>         Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>> 
>> 00:01.0 ISA bridge [0601]: Intel Corporation 82371SB PIIX3 ISA 
>> [Natoma/Triton II] [8086:7000]
>>         Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
>>         Physical Slot: 1
>>         Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>> 
>> 00:01.1 IDE interface [0101]: Intel Corporation 82371SB PIIX3 IDE 
>> [Natoma/Triton II] [8086:7010] (prog-if 80 [Master])
>>         Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
>>         Physical Slot: 1
>>         Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>>         Region 0: [virtual] Memory at 000001f0 (32-bit, non-prefetchable) 
>> [size=8]
>>         Region 1: [virtual] Memory at 000003f0 (type 3, non-prefetchable) 
>> [size=1]
>>         Region 2: [virtual] Memory at 00000170 (32-bit, non-prefetchable) 
>> [size=8]
>>         Region 3: [virtual] Memory at 00000370 (type 3, non-prefetchable) 
>> [size=1]
>>         Region 4: I/O ports at c260 [size=16]
>>         Kernel driver in use: PIIX_IDE
>> 
>> 00:01.2 USB controller [0c03]: Intel Corporation 82371SB PIIX3 USB 
>> [Natoma/Triton II] [8086:7020] (rev 01) (prog-if 00 [UHCI])
>>         Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
>>         Physical Slot: 1
>>         Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>>         Interrupt: pin D routed to IRQ 23
>>         Region 4: I/O ports at c240 [size=32]
>>         Kernel driver in use: uhci_hcd
>> 
>> 00:01.3 Bridge [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI 
>> [8086:7113] (rev 03)
>>         Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
>>         Physical Slot: 1
>>         Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>>         Interrupt: pin A routed to IRQ 9
>> 
>> 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device 
>> [5853:0001] (rev 01)
>>         Subsystem: XenSource, Inc. Xen Platform Device [5853:0001]
>>         Physical Slot: 2
>>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>>         Interrupt: pin A routed to IRQ 24
>>         Region 0: I/O ports at c000 [size=256]
>>         Region 1: Memory at f2000000 (32-bit, prefetchable) [size=16M]
>>         Kernel driver in use: xen-platform-pci
>> 
>> 00:03.0 VGA compatible controller [0300]: Cirrus Logic GD 5446 [1013:00b8] 
>> (prog-if 00 [VGA controller])
>>         Subsystem: Red Hat, Inc Device [1af4:1100]
>>         Physical Slot: 3
>>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>>         Region 0: Memory at f0000000 (32-bit, prefetchable) [size=32M]
>>         Region 1: Memory at f30b0000 (32-bit, non-prefetchable) [size=4K]
>>         Expansion ROM at f30a0000 [disabled] [size=64K]
>> 
>> 00:05.0 VGA compatible controller [0300]: Advanced Micro Devices [AMD] nee 
>> ATI Turks [Radeon HD 6570] [1002:6759] (prog-if 00 [VGA controller])
>>         Subsystem: PC Partner Limited Device [174b:e193]
>>         Physical Slot: 5
>>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
>> Stepping- SERR- FastB2B- DisINTx+
>>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>>         Interrupt: pin A routed to IRQ 81
>>         Region 0: Memory at e0000000 (64-bit, prefetchable) [size=256M]
>>         Region 2: Memory at f3060000 (64-bit, non-prefetchable) [size=128K]
>>         Region 4: I/O ports at c100 [size=256]
>>         Expansion ROM at f3080000 [disabled] [size=128K]
>>         Capabilities: [50] Power Management version 3
>>                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA 
>> PME(D0-,D1-,D2-,D3hot-,D3cold-)
>>                 Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
>>         Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
>>                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, 
>> L1 unlimited
>>                         ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
>>                 DevCtl: Report errors: Correctable- Non-Fatal- Fatal- 
>> Unsupported-
>>                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
>>                         MaxPayload 128 bytes, MaxReadReq 512 bytes
>>                 DevSta: CorrErr+ UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- 
>> TransPend-
>>                 LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, 
>> Latency L0 <64ns, L1 <1us
>>                         ClockPM- Surprise- LLActRep- BwNot-
>>                 LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- 
>> CommClk-
>>                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
>>                 LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ 
>> DLActive- BWMgmt- ABWMgmt-
>>                 DevCap2: Completion Timeout: Not Supported, TimeoutDis-
>>                 DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
>>                 LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- 
>> SpeedDis-, Selectable De-emphasis: -6dB
>>                          Transmit Margin: Normal Operating Range, 
>> EnterModifiedCompliance- ComplianceSOS-
>>                          Compliance De-emphasis: -6dB
>>                 LnkSta2: Current De-emphasis Level: -3.5dB, 
>> EqualizationComplete-, EqualizationPhase1-
>>                          EqualizationPhase2-, EqualizationPhase3-, 
>> LinkEqualizationRequest-
>>         Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
>>                 Address: 00000000fee57000  Data: 4300
>>         Capabilities: [100 v9] #1002
>>         Kernel driver in use: radeon





reply via email to

[Prev in Thread] Current Thread [Next in Thread]