Re: [Qemu-devel] usb-storage assertions

[Daniel P. Berrange]

On Tue, Jan 12, 2016 at 03:36:40PM +0100, Kevin Wolf wrote:
> Am 12.01.2016 um 15:17 hat Gerd Hoffmann geschrieben:
> > On Sa, 2016-01-09 at 20:34 +0300, Andrey Korolyov wrote:
> > > Hello,
> > > 
> > > during regular operations within linux guest with USB EHCI frontend I
> > > am seeing process crashes with an assert during regular operations
> > > like dpkg install:
> > > 
> > > hw/usb/dev-storage.c:334: usb_msd_handle_reset: Assertion `s->req ==
> > > ((void *)0)' failed.
> > > 
> > > This does happen when real block backend is an USB-attached device
> > > itself, so we are hitting some subtle race right there, because a
> > > single change of a backend to a raw file sitting on SSD has never
> > > triggered this block with the linux guest (but xBSD pre-reboot flush
> > > would trigger assertion from time to time in the usb_cancel_packet()
> > > without regarding physical backend type:
> > > 
> > > hw/usb/core.c:508: usb_cancel_packet: Assertion
> > > `usb_packet_is_inflight(p)' failed.
> > > )
> > > 
> > > Since the 2.2.0 which I am running is not exactly freshest one, I
> > > could re-check with master today or tomorrow, but log for hw/usb/ has
> > > no related commits for intentional fixes over one or another issue.
> > 
> > Checking would be nice, but I likewise don't expect things being
> > different on 2.5.
> > 
> > Most likely these are bugs in rarely taken code paths, probably because
> > the guest cancels requests and/or resets device due to timeouts (which
> > in turn are triggered by slow backing storage on the host).
> > 
> > Any chance you can rebuild qemu with DEBUG_MSD enabled (in
> > hw/usb/dev-storage.c), then trigger the asserts again, so we hopefully
> > get a useful trail of the events triggering the bugs?
> > 
> > Kevin, is is possible to limit transfer rates for block devices, to
> > simplify debugging stuff like this?  Preferably in a way supported by
> > libvirt?
> 
> Yes. In the qemu command line, you can limit the rate with -drive
> throttling.bps_total=... and similar options (you can instead throttle
> based on iops instead, and you can have separate read/write limits - I
> don't think you need any of these, though).
> 
> I have never used it in libvirt, but libvirt.org seems to suggest that
> it looks like this (seems to be for Xen, but you get the idea):
> 
>     <disk type='file' snapshot='external'>
>       <driver name="tap" type="aio" cache="default"/>
>       <source file='/var/lib/xen/images/fv0' startupPolicy='optional'>
>         <seclabel relabel='no'/>
>       </source>
>       <target dev='hda' bus='ide'/>
>       <iotune>
>         <total_bytes_sec>10000000</total_bytes_sec>
>         <read_iops_sec>400000</read_iops_sec>
>         <write_iops_sec>100000</write_iops_sec>
>       </iotune>
>     </disk>

Yes, that works with QEMU and will map through to the -drive I/O
tuning parameters.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|