Re: [Qemu-devel] Isuue assiging devices using VFIO on x86

[Bhushan Bharat-R65777]

> -----Original Message-----
> From: Alex Williamson [mailto:address@hidden
> Sent: Wednesday, August 29, 2012 12:16 PM
> To: Bhushan Bharat-R65777
> Cc: address@hidden; Avi Kivity; address@hidden
> Subject: Re: Isuue assiging devices using VFIO on x86
> 
> On Tue, 2012-08-28 at 17:10 +0000, Bhushan Bharat-R65777 wrote:
> >
> > > -----Original Message-----
> > > From: Alex Williamson [mailto:address@hidden
> > > Sent: Tuesday, August 28, 2012 9:27 PM
> > > To: Bhushan Bharat-R65777
> > > Cc: address@hidden; Avi Kivity; address@hidden
> > > Subject: Re: Isuue assiging devices using VFIO on x86
> > >
> > > On Tue, 2012-08-28 at 09:23 +0000, Bhushan Bharat-R65777 wrote:
> > > > Hi Alex,
> > > >
> > > > In my susyem I have following devices:
> > > >
> > > > I tried assigning a following PCI devices:
> > > > 00:03.0 Communication controller: Intel Corporation 4 Series
> > > > Chipset HECI
> > > Controller (rev 03)
> > > > 00:03.2 IDE interface: Intel Corporation 4 Series Chipset PT IDER
> > > > Controller
> > > (rev 03)
> > > > 00:03.3 Serial controller: Intel Corporation 4 Series Chipset
> > > > Serial KT
> > > Controller (rev 03)
> > > >
> > > > and getting below error:
> > > >
> > > > ---------------
> > > > Command:
> > > > -------
> > > > qemu-system-x86_64 -enable-kvm  -nographic  -kernel
> > > > /boot/vmlinuz-3.6.0-rc2+ -
> > > initrd /boot/initramfs-3.6.0-rc2+.img -append "root=/dev/sda1" -m
> > > 1024 -drive
> > > file=/home/kvmdev/debian_squeeze_amd64_standard.qcow2 -device vfio-
> > > pci,host=0000:00:03.0 -device vfio-pci,host=0000:00:03.2 -device
> > > vfio-
> > > pci,host=0000:00:03.3
> > > >
> > > > Error prints:
> > > > ----
> > > > qemu-system-x86_64: -device vfio-pci,host=0000:00:03.0: Warning,
> > > > device
> > > 0000:00:03.0 does not support reset
> > > >
> > > > qemu-system-x86_64: -device vfio-pci,host=0000:00:03.0: VFIO
> > > > 0000:00:03.0 BAR
> > > 0 is too small to mmap, this may affect performance.
> > > >
> > > > qemu-system-x86_64: -device vfio-pci,host=0000:00:03.2: Warning,
> > > > device
> > > 0000:00:03.2 does not support reset
> > > >
> > > > qemu-system-x86_64: -device vfio-pci,host=0000:00:03.3: Warning,
> > > > device
> > > 0000:00:03.3 does not support reset
> > > >
> > > > qemu: hardware error: register_ioport_read: invalid opaque for
> > > > address 0x3f6 CPU #0:
> > > > EAX=80000003 EBX=3ffe0e80 ECX=80000003 EDX=00000cfc ESI=00002800
> > > > EDI=80002804 EBP=00006fc0 ESP=00006f38
> > > > EIP=3ffec005 EFL=00000086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
> > > > ES =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
> > > > CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
> > > > SS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
> > > > DS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
> > > > FS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
> > > > GS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
> > > > LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT TR =0000 00000000
> > > > 0000ffff 00008b00 DPL=0 TSS32-busy
> > > > GDT=     000fcd68 00000037
> > > > IDT=     000fdb60 00000000
> > > > CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
> > > > DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> > > DR3=0000000000000000
> > > > DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000
> > > > FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
> > > > FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
> > > > FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
> > > > FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
> > > > FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
> > > > XMM00=00000000000000000000000000000000
> > > > XMM01=00000000000000000000000000000000
> > > > XMM02=00000000000000000000000000000000
> > > > XMM03=00000000000000000000000000000000
> > > > XMM04=00000000000000000000000000000000
> > > > XMM05=00000000000000000000000000000000
> > > > XMM06=00000000000000000000000000000000
> > > > XMM07=00000000000000000000000000000000
> > > > Aborted (core dumped)
> > > >
> > > > --------------------
> > > >
> > > > Linux: http://github.com/awilliam/linux-vfio.git next
> > > > Qemu: https://github.com/awilliam/qemu-vfio.git  iommu-group-vfio
> > > >
> > > > Any idea what's is the issue.
> > >
> > > Please try the vfio-pci-for-qemu-1.2-v3 qemu-vfio.git tag, I'm not
> > > even sure what's in that old iommu-group-vfio branch.  Thanks,
> >
> > Can you share the command to configure qemu?
> 
> Nothing special: ./configure
> 
> I sometimes set --prefix or --target-list, but nothing that changes the
> binaries.  Thanks,

Hello Alex,
I am using " ./configure --target-list=x86_64-softmmu --enable-kvm " .
QEMU repo is at:
commit 3b4672f6614e07f9dc0a85a12a8c89d480a2493c
Author: Alex Williamson <address@hidden>
Date:   Tue Aug 14 14:07:19 2012 -0600

    vfio: Enable vfio-pci and mark supported
    
    Signed-off-by: Alex Williamson <address@hidden>

Linux Repo is at :
commit d9875690d9b89a866022ff49e3fcea892345ad92
Author: Linus Torvalds <address@hidden>
Date:   Thu Aug 16 14:51:24 2012 -0700

    Linux 3.6-rc2

---------------------------

Then I bound 3 pci devices with VFIO and run below command to launch VM
qemu-system-x86_64 -enable-kvm  -nographic  -kernel /boot/vmlinuz-3.6.0-rc2+ 
-initrd /boot/initramfs-3.6.0-rc2+.img -append "root=/dev/sda1" -m 1024 -drive 
file=/home/kvmdev/debian_squeeze_amd64_standard.qcow2 -device 
vfio-pci,host=0000:00:03.0 -device vfio-pci,host=0000:00:03.2 -device 
vfio-pci,host=0000:00:03.3

Then I was getting below errors: 
qemu-system-x86_64: -device vfio-pci,host=0000:00:03.0: vfio: failed to set 
iommu for container: Operation not permitted

qemu-system-x86_64: -device vfio-pci,host=0000:00:03.0: vfio: failed to setup 
container for group 2

qemu-system-x86_64: -device vfio-pci,host=0000:00:03.0: vfio: failed to get 
group 2

qemu-system-x86_64: -device vfio-pci,host=0000:00:03.0: Device 'vfio-pci' could 
not be initialized
---------

Then I set /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts = 1

And getting below error:

qemu-system-x86_64: -device vfio-pci,host=0000:00:03.0: Warning, device 
0000:00:03.0 does not support reset

qemu-system-x86_64: -device vfio-pci,host=0000:00:03.2: Warning, device 
0000:00:03.2 does not support reset

qemu-system-x86_64: -device vfio-pci,host=0000:00:03.3: Warning, device 
0000:00:03.3 does not support reset

qemu: hardware error: register_ioport_read: invalid opaque for address 0x3f6
CPU #0:
EAX=80000003 EBX=3ffe0e80 ECX=80000003 EDX=00000cfc
ESI=00002800 EDI=80002804 EBP=00006fc0 ESP=00006f38
EIP=3ffec005 EFL=00000086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000fcd68 00000037
IDT=     000fdb60 00000000
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 
DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
Aborted (core dumped)

Thanks a lot for your help
-Bharat