qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH V4 0/6] pxb: fix 64-bit MMIO allocation


From: Marcel Apfelbaum
Subject: Re: [Qemu-devel] [PATCH V4 0/6] pxb: fix 64-bit MMIO allocation
Date: Fri, 1 Jul 2016 17:15:10 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0

On 07/01/2016 04:57 PM, Laszlo Ersek wrote:
On 06/30/16 21:07, Marcel Apfelbaum wrote:
64-bit BARs allocations fix for devices behind PXBs/PXB-PCIEs.

In build_crs() the calculation and merging of the ranges already happens
in 64-bit, but the entry boundaries are silently truncated to 32-bit in the
call to aml_dword_memory(). Fix it by handling the 64-bit MMIO ranges 
separately.

v3 -> v4:
  Addressed Igor's comments (thanks for the productive review!)
  - Split pxb test patch (previously patch 3/3) into the test itself (patch 
1/6) and the blobs (patch 6/6).
  - New patch declaring pxb/pxb-pxie as not hot-pluggable.
     - Note that it does not solve the DSDT issue, but it is a prerequisite for 
the next patch.
  - New patch solving the DSDT issue spotted by Igor.
      - Using V=1 DIFF=diff make check does make it easier to review the ACPI 
changes, thanks.
  - Patches 4 and 5 untouched (previously patches 1/3 and 2/3)

v2 -> v3:
  - split original series "pci: better support for 64-bit MMIO allocation" into 
2 series:
     - this is the first part dealing with correct 64-bit MMIO ACPI computation
     - the second one will include 64-bit MMIO reservation for PCI hotplug
  - Add pxb/pxb-pcie tests (Igor) - See diffs below (*)
  - Re-based on latest master.

v1 -> v2:
  - resolved some styling issues (Laszlo)
  - rebase on latest master (Laszlo)

Thank you,
Marcel

Marcel Apfelbaum (6):
   tests/acpi: add pxb/pxb-pcie tests
   hw/pxb: declare pxb devices as not hot-pluggable
   hw/acpi: fix a DSDT table issue when a pxb is present.
   acpi: refactor pxb crs computation
   hw/apci: handle 64-bit MMIO regions correctly
   tests/acpi: Add pxb/pxb-pcie tests blobs

  hw/i386/acpi-build.c                   | 131 ++++++++++++++++++++++++---------
  hw/pci-bridge/pci_expander_bridge.c    |   2 +
  tests/acpi-test-data/pc/DSDT.pxb       | Bin 0 -> 6280 bytes
  tests/acpi-test-data/q35/DSDT.pxb_pcie | Bin 0 -> 9098 bytes
  tests/bios-tables-test.c               |  37 ++++++++++
  5 files changed, 134 insertions(+), 36 deletions(-)
  create mode 100644 tests/acpi-test-data/pc/DSDT.pxb
  create mode 100644 tests/acpi-test-data/q35/DSDT.pxb_pcie


I used the following commands to test (part of) the series:

VIRTIO10=1
ISO=/mnt/data/isos/iso-fedora/Fedora-Workstation-Live-x86_64-24-1.2.iso
CODE=/home/virt-images/OVMF_CODE.fd
TMPL=/home/virt-images/OVMF_VARS.fd
TFTP=/var/lib/dnsmasq
BF=shim.efi

if [ $VIRTIO10 -eq 0 ]; then
   MODERN=disable-legacy=off,disable-modern=on
else
   MODERN=disable-legacy=on,disable-modern=off
fi

cp $TMPL vars3.fd

qemu-system-x86_64 \
   -m 2048 \
   -M pc \
   -enable-kvm \
   -device qxl-vga \
   -drive if=pflash,readonly,format=raw,file=$CODE \
   -drive if=pflash,format=raw,file=vars3.fd \
   -drive id=cdrom,if=none,readonly,format=raw,cache=writethrough,file=$ISO \
   \
   -debugcon file:debug3.log \
   -global isa-debugcon.iobase=0x402 \
   \
   -chardev stdio,signal=off,mux=on,id=char0 \
   -mon chardev=char0,mode=readline,default \
   -serial chardev:char0 \
   \
   -device pxb,bus=pci.0,id=bridge1,bus_nr=11 \
   -device pxb,bus=pci.0,id=bridge2,bus_nr=7 \
   -device pxb,bus=pci.0,id=bridge3,bus_nr=15 \
   \
   -device virtio-scsi-pci,id=scsi0,$MODERN,bus=pci.0 \
   -device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=1 \
   \
   -netdev 
user,id=netdev0,hostfwd=tcp:127.0.0.1:2227-:22,tftp=$TFTP,bootfile=$BF,net=10.0.2.0/24
 \
   -device 
virtio-net-pci,netdev=netdev0,romfile=,$MODERN,bootindex=2,bus=bridge1,addr=2 \
   \
   -netdev user,id=netdev1,hostfwd=tcp:127.0.0.1:2228-:22,net=10.0.3.0/24 \
   -device 
virtio-net-pci,netdev=netdev1,romfile=,$MODERN,bootindex=3,bus=bridge2,addr=3 \
   \
   -netdev user,id=netdev2,hostfwd=tcp:127.0.0.1:2229-:22,net=10.0.4.0/24 \
   -device 
virtio-net-pci,netdev=netdev2,romfile=,$MODERN,bootindex=4,bus=bridge3,addr=4 \
   \
   -global PIIX4_PM.disable_s3=0 \


This is /proc/iomem from within the guest:

00000000-00000fff : reserved
00001000-0009ffff : System RAM
000a0000-000bffff : PCI Bus 0000:00
000f0000-000fffff : System ROM
00100000-007fffff : System RAM
00800000-00807fff : ACPI Non-volatile Storage
00808000-0080ffff : System RAM
00810000-008fffff : ACPI Non-volatile Storage
00900000-7e529017 : System RAM
   02000000-027d3cf3 : Kernel code
   027d3cf4-02d4dfff : Kernel data
   02edc000-0305dfff : Kernel bss
7e529018-7e532857 : System RAM
7e532858-7e800fff : System RAM
7e801000-7e820fff : ACPI Non-volatile Storage
7e821000-7e84afff : reserved
7e84b000-7ebe1fff : System RAM
7ebe2000-7ebfdfff : reserved
7ebfe000-7ec22fff : System RAM
7ec23000-7ec35fff : reserved
7ec36000-7ec99fff : System RAM
7ec9a000-7ee16fff : reserved
7ee17000-7fe96fff : System RAM
7fe97000-7feeefff : reserved
7feef000-7fef6fff : ACPI Tables
7fef7000-7fefafff : ACPI Non-volatile Storage
7fefb000-7ffcffff : System RAM
7ffd0000-7ffeffff : reserved
7fff0000-7fffffff : System RAM
80000000-880fffff : PCI Bus 0000:00
   80000000-83ffffff : 0000:00:02.0
   84000000-87ffffff : 0000:00:02.0
   88000000-88001fff : 0000:00:02.0
   88002000-88002fff : 0000:00:06.0
   88010000-8801ffff : 0000:00:02.0
88100000-881fffff : PCI Bus 0000:07
   88100000-881fffff : PCI Bus 0000:08
     88100000-88100fff : 0000:08:03.0
88200000-882fffff : PCI Bus 0000:0b
   88200000-882fffff : PCI Bus 0000:0c
     88200000-88200fff : 0000:0c:02.0
88300000-883fffff : PCI Bus 0000:0f
   88300000-883fffff : PCI Bus 0000:10
     88300000-88300fff : 0000:10:04.0
88400000-febfffff : PCI Bus 0000:00
fec00000-fec003ff : IOAPIC 0
fed00000-fed003ff : HPET 0
   fed00000-fed003ff : PNP0103:00
fee00000-fee00fff : Local APIC
ffe00000-ffffffff : reserved
800000000-8007fffff : PCI Bus 0000:00
   800000000-8007fffff : 0000:00:06.0
     800000000-8007fffff : virtio-pci-modern
800800000-800ffffff : PCI Bus 0000:07
   800800000-800ffffff : PCI Bus 0000:08
     800800000-800ffffff : 0000:08:03.0
       800800000-800ffffff : virtio-pci-modern
801000000-8017fffff : PCI Bus 0000:0b
   801000000-8017fffff : PCI Bus 0000:0c
     801000000-8017fffff : 0000:0c:02.0
       801000000-8017fffff : virtio-pci-modern
801800000-801ffffff : PCI Bus 0000:0f
   801800000-801ffffff : PCI Bus 0000:10
     801800000-801ffffff : 0000:10:04.0
       801800000-801ffffff : virtio-pci-modern

It looks good to me. I also checked the _CRS objects in the DSDT in the guest 
(cursorily), it seemed okay.

Guest dmesg contains no problems.

The virtio-scsi CD obviously worked (booted Fedora 24 Live off of it). I also 
logged into the guest through all three NICs, with ssh.

Below an excerpt from the OVMF debug log -- it appears consistent with the 
guest's /proc/iomem:

PciHostBridgeGetRootBridges: 3 extra root buses reported by QEMU
InitRootBridge: populated root bus 0, with room for 6 subordinate bus(es)
InitRootBridge: populated root bus 7, with room for 3 subordinate bus(es)
InitRootBridge: populated root bus 11, with room for 3 subordinate bus(es)
InitRootBridge: populated root bus 15, with room for 240 subordinate bus(es)
RootBridge: PciRoot(0x0)
   Support/Attr: 70069 / 70069
     DmaAbove4G: No
NoExtConfSpace: Yes
      AllocAttr: 3 (CombineMemPMem Mem64Decode)
            Bus: 0 - 6
             Io: C000 - FFFF
            Mem: 80000000 - FBFFFFFF
     MemAbove4G: 800000000 - FFFFFFFFF
           PMem: FFFFFFFFFFFFFFFF - 0
    PMemAbove4G: FFFFFFFFFFFFFFFF - 0
PciHostBridgeDxe: IntersectMemoryDescriptor: add [800000000, 1000000000): 
Success
RootBridge: PciRoot(0x7)
   Support/Attr: 70069 / 70069
     DmaAbove4G: No
NoExtConfSpace: Yes
      AllocAttr: 3 (CombineMemPMem Mem64Decode)
            Bus: 7 - A
             Io: C000 - FFFF
            Mem: 80000000 - FBFFFFFF
     MemAbove4G: 800000000 - FFFFFFFFF
           PMem: FFFFFFFFFFFFFFFF - 0
    PMemAbove4G: FFFFFFFFFFFFFFFF - 0
RootBridge: PciRoot(0xB)
   Support/Attr: 70069 / 70069
     DmaAbove4G: No
NoExtConfSpace: Yes
      AllocAttr: 3 (CombineMemPMem Mem64Decode)
            Bus: B - E
             Io: C000 - FFFF
            Mem: 80000000 - FBFFFFFF
     MemAbove4G: 800000000 - FFFFFFFFF
           PMem: FFFFFFFFFFFFFFFF - 0
    PMemAbove4G: FFFFFFFFFFFFFFFF - 0
RootBridge: PciRoot(0xF)
   Support/Attr: 70069 / 70069
     DmaAbove4G: No
NoExtConfSpace: Yes
      AllocAttr: 3 (CombineMemPMem Mem64Decode)
            Bus: F - FF
             Io: C000 - FFFF
            Mem: 80000000 - FBFFFFFF
     MemAbove4G: 800000000 - FFFFFFFFF
           PMem: FFFFFFFFFFFFFFFF - 0
    PMemAbove4G: FFFFFFFFFFFFFFFF - 0

[...]

PCI Bus First Scanning
PciBus: Discovered PCI @ [00|00|00]

PciBus: Discovered PCI @ [00|01|00]

PciBus: Discovered PCI @ [00|01|01]
    BAR[4]: Type =   Io32; Alignment = 0xF;     Length = 0x10;  Offset = 0x20

PciBus: Discovered PCI @ [00|01|03]

PciBus: Discovered PCI @ [00|02|00]
    BAR[0]: Type =  Mem32; Alignment = 0x3FFFFFF;       Length = 0x4000000;     
Offset = 0x10
    BAR[1]: Type =  Mem32; Alignment = 0x3FFFFFF;       Length = 0x4000000;     
Offset = 0x14
    BAR[2]: Type =  Mem32; Alignment = 0x1FFF;  Length = 0x2000;        Offset 
= 0x18
    BAR[3]: Type =   Io32; Alignment = 0x1F;    Length = 0x20;  Offset = 0x1C

PciBus: Discovered PCI @ [00|03|00]

PciBus: Discovered PCI @ [00|04|00]

PciBus: Discovered PCI @ [00|05|00]

PciBus: Discovered PCI @ [00|06|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciBus: Discovered PPB @ [07|00|00]

PciBus: Discovered PCI @ [08|03|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciBus: Discovered PPB @ [0B|00|00]

PciBus: Discovered PCI @ [0C|02|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciBus: Discovered PPB @ [0F|00|00]

PciBus: Discovered PCI @ [10|04|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciBus: Discovered PCI @ [00|00|00]

PciBus: Discovered PCI @ [00|01|00]

PciBus: Discovered PCI @ [00|01|01]
    BAR[4]: Type =   Io32; Alignment = 0xF;     Length = 0x10;  Offset = 0x20

PciBus: Discovered PCI @ [00|01|03]

PciBus: Discovered PCI @ [00|02|00]
    BAR[0]: Type =  Mem32; Alignment = 0x3FFFFFF;       Length = 0x4000000;     
Offset = 0x10
    BAR[1]: Type =  Mem32; Alignment = 0x3FFFFFF;       Length = 0x4000000;     
Offset = 0x14
    BAR[2]: Type =  Mem32; Alignment = 0x1FFF;  Length = 0x2000;        Offset 
= 0x18
    BAR[3]: Type =   Io32; Alignment = 0x1F;    Length = 0x20;  Offset = 0x1C

PciBus: Discovered PCI @ [00|03|00]

PciBus: Discovered PCI @ [00|04|00]

PciBus: Discovered PCI @ [00|05|00]

PciBus: Discovered PCI @ [00|06|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciBus: Discovered PPB @ [07|00|00]

PciBus: Discovered PCI @ [08|03|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciBus: Discovered PPB @ [0B|00|00]

PciBus: Discovered PCI @ [0C|02|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciBus: Discovered PPB @ [0F|00|00]

PciBus: Discovered PCI @ [10|04|00]
    BAR[1]: Type =  Mem32; Alignment = 0xFFF;   Length = 0x1000;        Offset 
= 0x14
    BAR[4]: Type = PMem64; Alignment = 0x7FFFFF;        Length = 0x800000;      
Offset = 0x20

PciHostBridge: SubmitResources for PciRoot(0x0)
  I/O: Granularity/SpecificFlag = 0 / 01
       Length/Alignment = 0x1000 / 0xFFF
  Mem: Granularity/SpecificFlag = 32 / 00
       Length/Alignment = 0x8100000 / 0x3FFFFFF
  Mem: Granularity/SpecificFlag = 64 / 00
       Length/Alignment = 0x800000 / 0x7FFFFF
PciBus: HostBridge->SubmitResources() - Success
PciHostBridge: SubmitResources for PciRoot(0x7)
  Mem: Granularity/SpecificFlag = 32 / 00
       Length/Alignment = 0x100000 / 0xFFFFF
  Mem: Granularity/SpecificFlag = 64 / 00
       Length/Alignment = 0x800000 / 0x7FFFFF
PciBus: HostBridge->SubmitResources() - Success
PciHostBridge: SubmitResources for PciRoot(0xB)
  Mem: Granularity/SpecificFlag = 32 / 00
       Length/Alignment = 0x100000 / 0xFFFFF
  Mem: Granularity/SpecificFlag = 64 / 00
       Length/Alignment = 0x800000 / 0x7FFFFF
PciBus: HostBridge->SubmitResources() - Success
PciHostBridge: SubmitResources for PciRoot(0xF)
  Mem: Granularity/SpecificFlag = 32 / 00
       Length/Alignment = 0x100000 / 0xFFFFF
  Mem: Granularity/SpecificFlag = 64 / 00
       Length/Alignment = 0x800000 / 0x7FFFFF
PciBus: HostBridge->SubmitResources() - Success
PciHostBridge: NotifyPhase (AllocateResources)
  RootBridge: PciRoot(0x0)
   Mem: Base/Length/Alignment = 80000000/8100000/3FFFFFF - Success
   Mem64: Base/Length/Alignment = 800000000/800000/7FFFFF - Success
   I/O: Base/Length/Alignment = C000/1000/FFF - Success
  RootBridge: PciRoot(0x7)
   Mem64: Base/Length/Alignment = 800800000/800000/7FFFFF - Success
   Mem: Base/Length/Alignment = 88100000/100000/FFFFF - Success
  RootBridge: PciRoot(0xB)
   Mem64: Base/Length/Alignment = 801000000/800000/7FFFFF - Success
   Mem: Base/Length/Alignment = 88200000/100000/FFFFF - Success
  RootBridge: PciRoot(0xF)
   Mem64: Base/Length/Alignment = 801800000/800000/7FFFFF - Success
   Mem: Base/Length/Alignment = 88300000/100000/FFFFF - Success
PciBus: HostBridge->NotifyPhase(AllocateResources) - Success
PciBus: Resource Map for Root Bridge PciRoot(0x0)
Type =   Io16; Base = 0xC000;   Length = 0x1000;        Alignment = 0xFFF
    Base = 0xC000;      Length = 0x20;  Alignment = 0x1F;       Owner = PCI 
[00|02|00:1C]
    Base = 0xC020;      Length = 0x10;  Alignment = 0xF;        Owner = PCI 
[00|01|01:20]
Type =  Mem32; Base = 0x80000000;       Length = 0x8100000;     Alignment = 
0x3FFFFFF
    Base = 0x80000000;  Length = 0x4000000;     Alignment = 0x3FFFFFF;  Owner = 
PCI [00|02|00:14]
    Base = 0x84000000;  Length = 0x4000000;     Alignment = 0x3FFFFFF;  Owner = 
PCI [00|02|00:10]
    Base = 0x88000000;  Length = 0x2000;        Alignment = 0x1FFF;     Owner = 
PCI [00|02|00:18]
    Base = 0x88002000;  Length = 0x1000;        Alignment = 0xFFF;      Owner = 
PCI [00|06|00:14]
Type =  Mem64; Base = 0x800000000;      Length = 0x800000;      Alignment = 
0x7FFFFF
    Base = 0x800000000; Length = 0x800000;      Alignment = 0x7FFFFF;   Owner = 
PCI [00|06|00:20]; Type = PMem64

PciBus: Resource Map for Root Bridge PciRoot(0x7)
Type =  Mem32; Base = 0x88100000;       Length = 0x100000;      Alignment = 
0xFFFFF
    Base = 0x88100000;  Length = 0x100000;      Alignment = 0xFFFFF;    Owner = 
PPB [07|00|00:**]
Type =  Mem64; Base = 0x800800000;      Length = 0x800000;      Alignment = 
0x7FFFFF
    Base = 0x800800000; Length = 0x800000;      Alignment = 0x7FFFFF;   Owner = 
PPB [07|00|00:**]; Type = PMem64

PciBus: Resource Map for Bridge [07|00|00]
Type =  Mem32; Base = 0x88100000;       Length = 0x100000;      Alignment = 
0xFFFFF
    Base = 0x88100000;  Length = 0x1000;        Alignment = 0xFFF;      Owner = 
PCI [08|03|00:14]
Type = PMem64; Base = 0x800800000;      Length = 0x800000;      Alignment = 
0x7FFFFF
    Base = 0x800800000; Length = 0x800000;      Alignment = 0x7FFFFF;   Owner = 
PCI [08|03|00:20]

PciBus: Resource Map for Root Bridge PciRoot(0xB)
Type =  Mem32; Base = 0x88200000;       Length = 0x100000;      Alignment = 
0xFFFFF
    Base = 0x88200000;  Length = 0x100000;      Alignment = 0xFFFFF;    Owner = 
PPB [0B|00|00:**]
Type =  Mem64; Base = 0x801000000;      Length = 0x800000;      Alignment = 
0x7FFFFF
    Base = 0x801000000; Length = 0x800000;      Alignment = 0x7FFFFF;   Owner = 
PPB [0B|00|00:**]; Type = PMem64

PciBus: Resource Map for Bridge [0B|00|00]
Type =  Mem32; Base = 0x88200000;       Length = 0x100000;      Alignment = 
0xFFFFF
    Base = 0x88200000;  Length = 0x1000;        Alignment = 0xFFF;      Owner = 
PCI [0C|02|00:14]
Type = PMem64; Base = 0x801000000;      Length = 0x800000;      Alignment = 
0x7FFFFF
    Base = 0x801000000; Length = 0x800000;      Alignment = 0x7FFFFF;   Owner = 
PCI [0C|02|00:20]

PciBus: Resource Map for Root Bridge PciRoot(0xF)
Type =  Mem32; Base = 0x88300000;       Length = 0x100000;      Alignment = 
0xFFFFF
    Base = 0x88300000;  Length = 0x100000;      Alignment = 0xFFFFF;    Owner = 
PPB [0F|00|00:**]
Type =  Mem64; Base = 0x801800000;      Length = 0x800000;      Alignment = 
0x7FFFFF
    Base = 0x801800000; Length = 0x800000;      Alignment = 0x7FFFFF;   Owner = 
PPB [0F|00|00:**]; Type = PMem64

PciBus: Resource Map for Bridge [0F|00|00]
Type =  Mem32; Base = 0x88300000;       Length = 0x100000;      Alignment = 
0xFFFFF
    Base = 0x88300000;  Length = 0x1000;        Alignment = 0xFFF;      Owner = 
PCI [10|04|00:14]
Type = PMem64; Base = 0x801800000;      Length = 0x800000;      Alignment = 
0x7FFFFF
    Base = 0x801800000; Length = 0x800000;      Alignment = 0x7FFFFF;   Owner = 
PCI [10|04|00:20]

For patches #2 through #5:

Tested-by: Laszlo Ersek <address@hidden>

I also ran "make check". The "check-qtest-i386" step (like all others) succeeded, which 
should cover patch #1 and #6 via "bios-tables-test.c". For those as well:

Tested-by: Laszlo Ersek <address@hidden>

Very appreciated!

Also for patch #5 that actually solves the problem:
Reported-by: Laszlo Ersek <address@hidden>

Thanks,
Marcel


Thanks
Laszlo





reply via email to

[Prev in Thread] Current Thread [Next in Thread]