Re: [Qemu-devel] [Xen-devel] RESEND [Xen-unstable][Qemu-xen] HVM Guest reading of Expansion ROM from passthroughed PCI device returns data from emulated VGA rom

[Sander Eikelenboom]

Monday, September 23, 2013, 5:03:15 AM, you wrote:



>> -----Original Message-----
>> From: address@hidden
>> [mailto:address@hidden On Behalf Of Sander Eikelenboom
>> Sent: Sunday, September 22, 2013 11:01 PM
>> To: Pasi Kärkkäinen
>> Cc: Anthony PERARD; xen-devel; address@hidden; Stefano Stabellini
>> Subject: Re: [Xen-devel] RESEND [Xen-unstable][Qemu-xen] HVM Guest reading
>> of Expansion ROM from passthroughed PCI device returns data from emulated
>> VGA rom
>> 
>> 
>> Sunday, September 22, 2013, 3:02:45 PM, you wrote:
>> 
>> > On Tue, Sep 17, 2013 at 04:33:39PM +0200, Sander Eikelenboom wrote:
>> >> *RESEND* due to exceeding the mailinglists limit for attachment size.
>> >>
>> >> Hi,
>> >>
>> 
>> > Hello,
>> 
>> >> I'm trying to get secondary vga-passthrough on a HVM guest to work with a
>> AMD HD6570 and the native kernel radeon driver and kernel modesetting.
>> >> So the guest still gets the emulated stdvga or cirrus device(used in my 
>> >> case
>> here) as primary/boot vga adapter.
>> >>
>> >> - When i don't passthrough the radeon card, the linux native radeon driver
>> loads fine.
>> >> - When i do passtrough the device to a HVM with the same kernel:
>> >>   The driver in the guest tries to read the pci expansion rom from the
>> passthroughed device to get the vbios.
>> >>   The driver reports a successful read, but fails because it can't find 
>> >> the
>> right string at the right offset.
>> >>
>> 
>> > Did you try with qemu-dm-traditional aswell? Does it have the same
>> problem?
>> 
>> Hi Pasi,
>> 
>> Yes i did and yes the same problem.
>> From what i recall i used to have succes with vga passthrough with a 
>> secondary
>> vga card, but that was some time ago.
>> I don't know which of the components (xen, dom0 kernel, domU kernel, radeon
>> driver, qemu has changed in such a way that it fails to work now ...
>> 
>> 
>> But in the mean time i tried to debug it further and from what i can see:
>> - Only the io port en mem of the pci device are mapped through the 
>> hypervisor.
>> - The rom is not, (a hypercall to do the memory mapping is never made) i 
>> tried
>> several things to get it to do the mapping, but so far failt to do so.
>> - It seems to be a 64bit capable device, some code comments and git commit
>> messages seem to suggest that there were/are some problems with that (in
>> the way the involved components interact)
>> 

> Can you send out the all the logs when using qemu-dm-traditional as well as 
> the guest configuration file used by xl ?

Sure, here is the complete set for qemu-dm-traditional.
I have added some extra printk's (see xen.diff and qemu.diff).

From what i see when combining the info from all logs, both ioports and iomem 
for the device (host 07:00.0 and 07:00.1, guest 00:05.0 and 00:06.0)  get 
mapped, but the expansion rom is not (not by hvmloader and not by qemu) as one 
can see in
xl dmesg output.
So qemu seems to create the Memoryregion for the rom, but it doesn't seem to be 
backed up by any mapping (at least not one that passes through 
XEN_DOMCTL_memory_mapping hypercall).
( how it exactly ends up pointing to the bios from the emulated cirrus i don't 
know )

from the qemu log i see a lot of my extra printk's hit for the expansion rom 
like:
pt_bar_mapping_one: [00:05:0] pt_bar_mapping_one: [Region:6] need unmapping in 
case I/O Space or Memory Space disable
pt_bar_mapping_one: [00:05:0] pt_bar_mapping_one: [Region:6] update bar mapping 
NOT needed [Address:ffffffffh] [Size:00020000h] [Type:00000008h]

I also don't know if it would be desirable to do a direct mapping, or that the 
guest should just get a copy at the right address of the memoryregion that is 
created by hvmloader/qemu ?

--
Sander

P.S.:
    The complete qemu log is a bit long for the mailinglist, so attached is 
only the first part.
    The complete log is downloadable from: 
http://www.eikelenboom.it/qemu-dm-guest.log



>> --
>> Sander
>> 
>> > Thanks,
>> 
>> > -- Pasi
>> 
>> >> So I inspected the rom by using sysfs with:
>> >> echo 1 > /sys/bus/pci/devices/<BDF>/rom
>> >> cat /sys/bus/pci/devices/<BDF>/rom
>> >>
>> >> - When i use this in dom0 (so without passthrough) i the contents of the
>> ROM are valid (as expected since the driver loads fine)
>> >> - When i use this in the Guest (passthrouhed), the contents of the ROM i 
>> >> get
>> are not from the passedthrough adapter, but from the emulated cirrus card

dmesg-dom0.txt
Description: Text document

dmesg-guest.txt
Description: Text document

guest.cfg
Description: Binary data

lspci-dom0.txt
Description: Text document

lspci-guest.txt
Description: Text document

qemu.diff
Description: Binary data

qemu-dm-guest-short.log
Description: Binary data

xen.diff
Description: Binary data

xl-dmesg-dom0.txt
Description: Text document

xl-info.txt
Description: Text document