[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH V4 0/6] pxb: fix 64-bit MMIO allocation
From: |
Laszlo Ersek |
Subject: |
Re: [Qemu-devel] [PATCH V4 0/6] pxb: fix 64-bit MMIO allocation |
Date: |
Fri, 1 Jul 2016 15:57:21 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 |
On 06/30/16 21:07, Marcel Apfelbaum wrote:
> 64-bit BARs allocations fix for devices behind PXBs/PXB-PCIEs.
>
> In build_crs() the calculation and merging of the ranges already happens
> in 64-bit, but the entry boundaries are silently truncated to 32-bit in the
> call to aml_dword_memory(). Fix it by handling the 64-bit MMIO ranges
> separately.
>
> v3 -> v4:
> Addressed Igor's comments (thanks for the productive review!)
> - Split pxb test patch (previously patch 3/3) into the test itself (patch
> 1/6) and the blobs (patch 6/6).
> - New patch declaring pxb/pxb-pxie as not hot-pluggable.
> - Note that it does not solve the DSDT issue, but it is a prerequisite
> for the next patch.
> - New patch solving the DSDT issue spotted by Igor.
> - Using V=1 DIFF=diff make check does make it easier to review the ACPI
> changes, thanks.
> - Patches 4 and 5 untouched (previously patches 1/3 and 2/3)
>
> v2 -> v3:
> - split original series "pci: better support for 64-bit MMIO allocation"
> into 2 series:
> - this is the first part dealing with correct 64-bit MMIO ACPI computation
> - the second one will include 64-bit MMIO reservation for PCI hotplug
> - Add pxb/pxb-pcie tests (Igor) - See diffs below (*)
> - Re-based on latest master.
>
> v1 -> v2:
> - resolved some styling issues (Laszlo)
> - rebase on latest master (Laszlo)
>
> Thank you,
> Marcel
>
> Marcel Apfelbaum (6):
> tests/acpi: add pxb/pxb-pcie tests
> hw/pxb: declare pxb devices as not hot-pluggable
> hw/acpi: fix a DSDT table issue when a pxb is present.
> acpi: refactor pxb crs computation
> hw/apci: handle 64-bit MMIO regions correctly
> tests/acpi: Add pxb/pxb-pcie tests blobs
>
> hw/i386/acpi-build.c | 131
> ++++++++++++++++++++++++---------
> hw/pci-bridge/pci_expander_bridge.c | 2 +
> tests/acpi-test-data/pc/DSDT.pxb | Bin 0 -> 6280 bytes
> tests/acpi-test-data/q35/DSDT.pxb_pcie | Bin 0 -> 9098 bytes
> tests/bios-tables-test.c | 37 ++++++++++
> 5 files changed, 134 insertions(+), 36 deletions(-)
> create mode 100644 tests/acpi-test-data/pc/DSDT.pxb
> create mode 100644 tests/acpi-test-data/q35/DSDT.pxb_pcie
>
I used the following commands to test (part of) the series:
> VIRTIO10=1
> ISO=/mnt/data/isos/iso-fedora/Fedora-Workstation-Live-x86_64-24-1.2.iso
> CODE=/home/virt-images/OVMF_CODE.fd
> TMPL=/home/virt-images/OVMF_VARS.fd
> TFTP=/var/lib/dnsmasq
> BF=shim.efi
>
> if [ $VIRTIO10 -eq 0 ]; then
> MODERN=disable-legacy=off,disable-modern=on
> else
> MODERN=disable-legacy=on,disable-modern=off
> fi
>
> cp $TMPL vars3.fd
>
> qemu-system-x86_64 \
> -m 2048 \
> -M pc \
> -enable-kvm \
> -device qxl-vga \
> -drive if=pflash,readonly,format=raw,file=$CODE \
> -drive if=pflash,format=raw,file=vars3.fd \
> -drive id=cdrom,if=none,readonly,format=raw,cache=writethrough,file=$ISO \
> \
> -debugcon file:debug3.log \
> -global isa-debugcon.iobase=0x402 \
> \
> -chardev stdio,signal=off,mux=on,id=char0 \
> -mon chardev=char0,mode=readline,default \
> -serial chardev:char0 \
> \
> -device pxb,bus=pci.0,id=bridge1,bus_nr=11 \
> -device pxb,bus=pci.0,id=bridge2,bus_nr=7 \
> -device pxb,bus=pci.0,id=bridge3,bus_nr=15 \
> \
> -device virtio-scsi-pci,id=scsi0,$MODERN,bus=pci.0 \
> -device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=1 \
> \
> -netdev
> user,id=netdev0,hostfwd=tcp:127.0.0.1:2227-:22,tftp=$TFTP,bootfile=$BF,net=10.0.2.0/24
> \
> -device
> virtio-net-pci,netdev=netdev0,romfile=,$MODERN,bootindex=2,bus=bridge1,addr=2
> \
> \
> -netdev user,id=netdev1,hostfwd=tcp:127.0.0.1:2228-:22,net=10.0.3.0/24 \
> -device
> virtio-net-pci,netdev=netdev1,romfile=,$MODERN,bootindex=3,bus=bridge2,addr=3
> \
> \
> -netdev user,id=netdev2,hostfwd=tcp:127.0.0.1:2229-:22,net=10.0.4.0/24 \
> -device
> virtio-net-pci,netdev=netdev2,romfile=,$MODERN,bootindex=4,bus=bridge3,addr=4
> \
> \
> -global PIIX4_PM.disable_s3=0 \
>
This is /proc/iomem from within the guest:
> 00000000-00000fff : reserved
> 00001000-0009ffff : System RAM
> 000a0000-000bffff : PCI Bus 0000:00
> 000f0000-000fffff : System ROM
> 00100000-007fffff : System RAM
> 00800000-00807fff : ACPI Non-volatile Storage
> 00808000-0080ffff : System RAM
> 00810000-008fffff : ACPI Non-volatile Storage
> 00900000-7e529017 : System RAM
> 02000000-027d3cf3 : Kernel code
> 027d3cf4-02d4dfff : Kernel data
> 02edc000-0305dfff : Kernel bss
> 7e529018-7e532857 : System RAM
> 7e532858-7e800fff : System RAM
> 7e801000-7e820fff : ACPI Non-volatile Storage
> 7e821000-7e84afff : reserved
> 7e84b000-7ebe1fff : System RAM
> 7ebe2000-7ebfdfff : reserved
> 7ebfe000-7ec22fff : System RAM
> 7ec23000-7ec35fff : reserved
> 7ec36000-7ec99fff : System RAM
> 7ec9a000-7ee16fff : reserved
> 7ee17000-7fe96fff : System RAM
> 7fe97000-7feeefff : reserved
> 7feef000-7fef6fff : ACPI Tables
> 7fef7000-7fefafff : ACPI Non-volatile Storage
> 7fefb000-7ffcffff : System RAM
> 7ffd0000-7ffeffff : reserved
> 7fff0000-7fffffff : System RAM
> 80000000-880fffff : PCI Bus 0000:00
> 80000000-83ffffff : 0000:00:02.0
> 84000000-87ffffff : 0000:00:02.0
> 88000000-88001fff : 0000:00:02.0
> 88002000-88002fff : 0000:00:06.0
> 88010000-8801ffff : 0000:00:02.0
> 88100000-881fffff : PCI Bus 0000:07
> 88100000-881fffff : PCI Bus 0000:08
> 88100000-88100fff : 0000:08:03.0
> 88200000-882fffff : PCI Bus 0000:0b
> 88200000-882fffff : PCI Bus 0000:0c
> 88200000-88200fff : 0000:0c:02.0
> 88300000-883fffff : PCI Bus 0000:0f
> 88300000-883fffff : PCI Bus 0000:10
> 88300000-88300fff : 0000:10:04.0
> 88400000-febfffff : PCI Bus 0000:00
> fec00000-fec003ff : IOAPIC 0
> fed00000-fed003ff : HPET 0
> fed00000-fed003ff : PNP0103:00
> fee00000-fee00fff : Local APIC
> ffe00000-ffffffff : reserved
> 800000000-8007fffff : PCI Bus 0000:00
> 800000000-8007fffff : 0000:00:06.0
> 800000000-8007fffff : virtio-pci-modern
> 800800000-800ffffff : PCI Bus 0000:07
> 800800000-800ffffff : PCI Bus 0000:08
> 800800000-800ffffff : 0000:08:03.0
> 800800000-800ffffff : virtio-pci-modern
> 801000000-8017fffff : PCI Bus 0000:0b
> 801000000-8017fffff : PCI Bus 0000:0c
> 801000000-8017fffff : 0000:0c:02.0
> 801000000-8017fffff : virtio-pci-modern
> 801800000-801ffffff : PCI Bus 0000:0f
> 801800000-801ffffff : PCI Bus 0000:10
> 801800000-801ffffff : 0000:10:04.0
> 801800000-801ffffff : virtio-pci-modern
It looks good to me. I also checked the _CRS objects in the DSDT in the guest
(cursorily), it seemed okay.
Guest dmesg contains no problems.
The virtio-scsi CD obviously worked (booted Fedora 24 Live off of it). I also
logged into the guest through all three NICs, with ssh.
Below an excerpt from the OVMF debug log -- it appears consistent with the
guest's /proc/iomem:
> PciHostBridgeGetRootBridges: 3 extra root buses reported by QEMU
> InitRootBridge: populated root bus 0, with room for 6 subordinate bus(es)
> InitRootBridge: populated root bus 7, with room for 3 subordinate bus(es)
> InitRootBridge: populated root bus 11, with room for 3 subordinate bus(es)
> InitRootBridge: populated root bus 15, with room for 240 subordinate bus(es)
> RootBridge: PciRoot(0x0)
> Support/Attr: 70069 / 70069
> DmaAbove4G: No
> NoExtConfSpace: Yes
> AllocAttr: 3 (CombineMemPMem Mem64Decode)
> Bus: 0 - 6
> Io: C000 - FFFF
> Mem: 80000000 - FBFFFFFF
> MemAbove4G: 800000000 - FFFFFFFFF
> PMem: FFFFFFFFFFFFFFFF - 0
> PMemAbove4G: FFFFFFFFFFFFFFFF - 0
> PciHostBridgeDxe: IntersectMemoryDescriptor: add [800000000, 1000000000):
> Success
> RootBridge: PciRoot(0x7)
> Support/Attr: 70069 / 70069
> DmaAbove4G: No
> NoExtConfSpace: Yes
> AllocAttr: 3 (CombineMemPMem Mem64Decode)
> Bus: 7 - A
> Io: C000 - FFFF
> Mem: 80000000 - FBFFFFFF
> MemAbove4G: 800000000 - FFFFFFFFF
> PMem: FFFFFFFFFFFFFFFF - 0
> PMemAbove4G: FFFFFFFFFFFFFFFF - 0
> RootBridge: PciRoot(0xB)
> Support/Attr: 70069 / 70069
> DmaAbove4G: No
> NoExtConfSpace: Yes
> AllocAttr: 3 (CombineMemPMem Mem64Decode)
> Bus: B - E
> Io: C000 - FFFF
> Mem: 80000000 - FBFFFFFF
> MemAbove4G: 800000000 - FFFFFFFFF
> PMem: FFFFFFFFFFFFFFFF - 0
> PMemAbove4G: FFFFFFFFFFFFFFFF - 0
> RootBridge: PciRoot(0xF)
> Support/Attr: 70069 / 70069
> DmaAbove4G: No
> NoExtConfSpace: Yes
> AllocAttr: 3 (CombineMemPMem Mem64Decode)
> Bus: F - FF
> Io: C000 - FFFF
> Mem: 80000000 - FBFFFFFF
> MemAbove4G: 800000000 - FFFFFFFFF
> PMem: FFFFFFFFFFFFFFFF - 0
> PMemAbove4G: FFFFFFFFFFFFFFFF - 0
>
> [...]
>
> PCI Bus First Scanning
> PciBus: Discovered PCI @ [00|00|00]
>
> PciBus: Discovered PCI @ [00|01|00]
>
> PciBus: Discovered PCI @ [00|01|01]
> BAR[4]: Type = Io32; Alignment = 0xF; Length = 0x10; Offset = 0x20
>
> PciBus: Discovered PCI @ [00|01|03]
>
> PciBus: Discovered PCI @ [00|02|00]
> BAR[0]: Type = Mem32; Alignment = 0x3FFFFFF; Length = 0x4000000;
> Offset = 0x10
> BAR[1]: Type = Mem32; Alignment = 0x3FFFFFF; Length = 0x4000000;
> Offset = 0x14
> BAR[2]: Type = Mem32; Alignment = 0x1FFF; Length = 0x2000; Offset
> = 0x18
> BAR[3]: Type = Io32; Alignment = 0x1F; Length = 0x20; Offset = 0x1C
>
> PciBus: Discovered PCI @ [00|03|00]
>
> PciBus: Discovered PCI @ [00|04|00]
>
> PciBus: Discovered PCI @ [00|05|00]
>
> PciBus: Discovered PCI @ [00|06|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciBus: Discovered PPB @ [07|00|00]
>
> PciBus: Discovered PCI @ [08|03|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciBus: Discovered PPB @ [0B|00|00]
>
> PciBus: Discovered PCI @ [0C|02|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciBus: Discovered PPB @ [0F|00|00]
>
> PciBus: Discovered PCI @ [10|04|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciBus: Discovered PCI @ [00|00|00]
>
> PciBus: Discovered PCI @ [00|01|00]
>
> PciBus: Discovered PCI @ [00|01|01]
> BAR[4]: Type = Io32; Alignment = 0xF; Length = 0x10; Offset = 0x20
>
> PciBus: Discovered PCI @ [00|01|03]
>
> PciBus: Discovered PCI @ [00|02|00]
> BAR[0]: Type = Mem32; Alignment = 0x3FFFFFF; Length = 0x4000000;
> Offset = 0x10
> BAR[1]: Type = Mem32; Alignment = 0x3FFFFFF; Length = 0x4000000;
> Offset = 0x14
> BAR[2]: Type = Mem32; Alignment = 0x1FFF; Length = 0x2000; Offset
> = 0x18
> BAR[3]: Type = Io32; Alignment = 0x1F; Length = 0x20; Offset = 0x1C
>
> PciBus: Discovered PCI @ [00|03|00]
>
> PciBus: Discovered PCI @ [00|04|00]
>
> PciBus: Discovered PCI @ [00|05|00]
>
> PciBus: Discovered PCI @ [00|06|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciBus: Discovered PPB @ [07|00|00]
>
> PciBus: Discovered PCI @ [08|03|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciBus: Discovered PPB @ [0B|00|00]
>
> PciBus: Discovered PCI @ [0C|02|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciBus: Discovered PPB @ [0F|00|00]
>
> PciBus: Discovered PCI @ [10|04|00]
> BAR[1]: Type = Mem32; Alignment = 0xFFF; Length = 0x1000; Offset
> = 0x14
> BAR[4]: Type = PMem64; Alignment = 0x7FFFFF; Length = 0x800000;
> Offset = 0x20
>
> PciHostBridge: SubmitResources for PciRoot(0x0)
> I/O: Granularity/SpecificFlag = 0 / 01
> Length/Alignment = 0x1000 / 0xFFF
> Mem: Granularity/SpecificFlag = 32 / 00
> Length/Alignment = 0x8100000 / 0x3FFFFFF
> Mem: Granularity/SpecificFlag = 64 / 00
> Length/Alignment = 0x800000 / 0x7FFFFF
> PciBus: HostBridge->SubmitResources() - Success
> PciHostBridge: SubmitResources for PciRoot(0x7)
> Mem: Granularity/SpecificFlag = 32 / 00
> Length/Alignment = 0x100000 / 0xFFFFF
> Mem: Granularity/SpecificFlag = 64 / 00
> Length/Alignment = 0x800000 / 0x7FFFFF
> PciBus: HostBridge->SubmitResources() - Success
> PciHostBridge: SubmitResources for PciRoot(0xB)
> Mem: Granularity/SpecificFlag = 32 / 00
> Length/Alignment = 0x100000 / 0xFFFFF
> Mem: Granularity/SpecificFlag = 64 / 00
> Length/Alignment = 0x800000 / 0x7FFFFF
> PciBus: HostBridge->SubmitResources() - Success
> PciHostBridge: SubmitResources for PciRoot(0xF)
> Mem: Granularity/SpecificFlag = 32 / 00
> Length/Alignment = 0x100000 / 0xFFFFF
> Mem: Granularity/SpecificFlag = 64 / 00
> Length/Alignment = 0x800000 / 0x7FFFFF
> PciBus: HostBridge->SubmitResources() - Success
> PciHostBridge: NotifyPhase (AllocateResources)
> RootBridge: PciRoot(0x0)
> Mem: Base/Length/Alignment = 80000000/8100000/3FFFFFF - Success
> Mem64: Base/Length/Alignment = 800000000/800000/7FFFFF - Success
> I/O: Base/Length/Alignment = C000/1000/FFF - Success
> RootBridge: PciRoot(0x7)
> Mem64: Base/Length/Alignment = 800800000/800000/7FFFFF - Success
> Mem: Base/Length/Alignment = 88100000/100000/FFFFF - Success
> RootBridge: PciRoot(0xB)
> Mem64: Base/Length/Alignment = 801000000/800000/7FFFFF - Success
> Mem: Base/Length/Alignment = 88200000/100000/FFFFF - Success
> RootBridge: PciRoot(0xF)
> Mem64: Base/Length/Alignment = 801800000/800000/7FFFFF - Success
> Mem: Base/Length/Alignment = 88300000/100000/FFFFF - Success
> PciBus: HostBridge->NotifyPhase(AllocateResources) - Success
> PciBus: Resource Map for Root Bridge PciRoot(0x0)
> Type = Io16; Base = 0xC000; Length = 0x1000; Alignment = 0xFFF
> Base = 0xC000; Length = 0x20; Alignment = 0x1F; Owner = PCI
> [00|02|00:1C]
> Base = 0xC020; Length = 0x10; Alignment = 0xF; Owner = PCI
> [00|01|01:20]
> Type = Mem32; Base = 0x80000000; Length = 0x8100000; Alignment =
> 0x3FFFFFF
> Base = 0x80000000; Length = 0x4000000; Alignment = 0x3FFFFFF; Owner =
> PCI [00|02|00:14]
> Base = 0x84000000; Length = 0x4000000; Alignment = 0x3FFFFFF; Owner =
> PCI [00|02|00:10]
> Base = 0x88000000; Length = 0x2000; Alignment = 0x1FFF; Owner =
> PCI [00|02|00:18]
> Base = 0x88002000; Length = 0x1000; Alignment = 0xFFF; Owner =
> PCI [00|06|00:14]
> Type = Mem64; Base = 0x800000000; Length = 0x800000; Alignment =
> 0x7FFFFF
> Base = 0x800000000; Length = 0x800000; Alignment = 0x7FFFFF;
> Owner = PCI [00|06|00:20]; Type = PMem64
>
> PciBus: Resource Map for Root Bridge PciRoot(0x7)
> Type = Mem32; Base = 0x88100000; Length = 0x100000; Alignment =
> 0xFFFFF
> Base = 0x88100000; Length = 0x100000; Alignment = 0xFFFFF; Owner =
> PPB [07|00|00:**]
> Type = Mem64; Base = 0x800800000; Length = 0x800000; Alignment =
> 0x7FFFFF
> Base = 0x800800000; Length = 0x800000; Alignment = 0x7FFFFF;
> Owner = PPB [07|00|00:**]; Type = PMem64
>
> PciBus: Resource Map for Bridge [07|00|00]
> Type = Mem32; Base = 0x88100000; Length = 0x100000; Alignment =
> 0xFFFFF
> Base = 0x88100000; Length = 0x1000; Alignment = 0xFFF; Owner =
> PCI [08|03|00:14]
> Type = PMem64; Base = 0x800800000; Length = 0x800000; Alignment =
> 0x7FFFFF
> Base = 0x800800000; Length = 0x800000; Alignment = 0x7FFFFF;
> Owner = PCI [08|03|00:20]
>
> PciBus: Resource Map for Root Bridge PciRoot(0xB)
> Type = Mem32; Base = 0x88200000; Length = 0x100000; Alignment =
> 0xFFFFF
> Base = 0x88200000; Length = 0x100000; Alignment = 0xFFFFF; Owner =
> PPB [0B|00|00:**]
> Type = Mem64; Base = 0x801000000; Length = 0x800000; Alignment =
> 0x7FFFFF
> Base = 0x801000000; Length = 0x800000; Alignment = 0x7FFFFF;
> Owner = PPB [0B|00|00:**]; Type = PMem64
>
> PciBus: Resource Map for Bridge [0B|00|00]
> Type = Mem32; Base = 0x88200000; Length = 0x100000; Alignment =
> 0xFFFFF
> Base = 0x88200000; Length = 0x1000; Alignment = 0xFFF; Owner =
> PCI [0C|02|00:14]
> Type = PMem64; Base = 0x801000000; Length = 0x800000; Alignment =
> 0x7FFFFF
> Base = 0x801000000; Length = 0x800000; Alignment = 0x7FFFFF;
> Owner = PCI [0C|02|00:20]
>
> PciBus: Resource Map for Root Bridge PciRoot(0xF)
> Type = Mem32; Base = 0x88300000; Length = 0x100000; Alignment =
> 0xFFFFF
> Base = 0x88300000; Length = 0x100000; Alignment = 0xFFFFF; Owner =
> PPB [0F|00|00:**]
> Type = Mem64; Base = 0x801800000; Length = 0x800000; Alignment =
> 0x7FFFFF
> Base = 0x801800000; Length = 0x800000; Alignment = 0x7FFFFF;
> Owner = PPB [0F|00|00:**]; Type = PMem64
>
> PciBus: Resource Map for Bridge [0F|00|00]
> Type = Mem32; Base = 0x88300000; Length = 0x100000; Alignment =
> 0xFFFFF
> Base = 0x88300000; Length = 0x1000; Alignment = 0xFFF; Owner =
> PCI [10|04|00:14]
> Type = PMem64; Base = 0x801800000; Length = 0x800000; Alignment =
> 0x7FFFFF
> Base = 0x801800000; Length = 0x800000; Alignment = 0x7FFFFF;
> Owner = PCI [10|04|00:20]
For patches #2 through #5:
Tested-by: Laszlo Ersek <address@hidden>
I also ran "make check". The "check-qtest-i386" step (like all others)
succeeded, which should cover patch #1 and #6 via "bios-tables-test.c". For
those as well:
Tested-by: Laszlo Ersek <address@hidden>
Thanks
Laszlo
- Re: [Qemu-devel] [PATCH V4 0/6] pxb: fix 64-bit MMIO allocation,
Laszlo Ersek <=