Re: [Qemu-discuss] How to resolve "Failed to mmap" error?

[Alex Williamson]

On Thu, 6 Oct 2016 21:07:06 -0700
A223 A223 <address@hidden> wrote:

> On Thu, Oct 6, 2016 at 8:30 PM, A223 A223 <address@hidden> wrote:
> > On Thu, Oct 6, 2016 at 9:46 AM, Alex Williamson
> > <address@hidden> wrote:  
> >> On Wed, 5 Oct 2016 20:30:34 -0700
> >> A223 A223 <address@hidden> wrote:
> >>  
> >>> On Wed, Oct 5, 2016 at 9:33 AM, Alex Williamson
> >>> <address@hidden> wrote:  
> >>> > On Wed, 5 Oct 2016 02:13:21 -0700
> >>> > A223 A223 <address@hidden> wrote:  
> >>> >> How can I go about tracking down the root cause of this error?  
> >>> >
> >>> > This often means that the device resources are in use by another,
> >>> > non-pci driver.  Look in /proc/iomem and /proc/ioport to see if any
> >>> > drivers are claiming sub-regions on the device.  You can also turn on
> >>> > tracing and look for the event vfio_region_mmap_fault, which might give
> >>> > us more information (see docs/tracing.txt).  Also, please provide 'sudo
> >>> > lspci -vvvs 5:00.0' so we can see the device resources.  Thanks,  
> >>>
> >>> The simpletrace output wouldn't parse for me using the python script
> >>> (I'll paste the trace at the end of this email), so I used the ftrace
> >>> backend instead (
> >>> https://drive.google.com/open?id=0Bwjufq6oAZMfMVpUYTUxODR0NW8 ). It's
> >>> quite large and I'm not sure what I'm looking for, so I didn't look at
> >>> it very carefully.  
> >>
> >> That's not the right trace, that's like kvm trace or something.  Try:
> >>
> >> # ./configure --enable-trace-backends=log
> >>
> >> # echo vfio_region_mmap_fault > events.txt
> >>
> >> Then add to your qemu commandline "-trace events=events.txt".  You
> >> should get maybe a couple extra log lines per boot, not gobs of output.  
> >
> > Ah, got it. Here are the relevant log lines then:
> >
> > address@hidden:vfio_region_mmap_fault Region 0000:05:00.0 BAR
> > 2 mmaps[0], [20000000000 - 1ffffffffff], fault: -22
> > qemu-system-x86_64: -device
> > vfio-pci,host=05:00.0,bus=root.1,addr=00.5: Failed to mmap
> > 0000:05:00.0 BAR 2. Performance may be slow  
> 
> I took a look at everything again and noted this in the lspci output
> for the Fresco Logic USB controller:
> Vector table: BAR=2 offset=00000000
> PBA: BAR=4 offset=00000000
> 
> Is it normal for both the Vector table and PBA to have a 0 offset?

Note that they're in different BARs, so it's a little unexpected but
it's not wrong.  The MSI-X table does appear to be the problem though.
Based on that trace line, the region size we're trying to mmap is
zero.  We're probably hitting a corner case of a page sized BAR
containing the MSI-X table, where we cannot mmap the page covering the
table.  We need to figure out where that's coming from.  Please add the
additional trace lines to your events.txt and try again:

vfio_region_sparse_mmap_header
vfio_region_sparse_mmap_entry
vfio_region_setup
vfio_msix_fixup

The bad news is that due to the MSI-X table, we're not going to be able
to mmap that BAR, the good news is that it probably doesn't actually
hurt performance.  With the table and PBA in separate BARs, my guess
would be that's the only thing in those BARs.  Thanks,

Alex