Re: [Qemu-devel] is there a limit on the number of in-flight I/O operations?

[Chris Friesen]

On 07/18/2014 02:13 PM, Paolo Bonzini wrote:
> Il 18/07/2014 18:22, Chris Friesen ha scritto:
> > On 07/18/2014 09:24 AM, Paolo Bonzini wrote:
> > > Il 18/07/2014 16:58, Chris Friesen ha scritto:
> > > > I've recently run up against an interesting issue where I had a number
> > > > of guests running and when I started doing heavy disk I/O on a virtio
> > > > disk (backed via ceph rbd) the memory consumption spiked and triggered
> > > > the OOM-killer.
> > > >
> > > > I want to reserve some memory for I/O, but I don't know how much it can
> > > > use in the worst-case.
> > > >
> > > > Is there a limit on the number of in-flight I/O operations?  (Preferably
> > > > as a configurable option, but even hard-coded would be good to know as
> > > > well.)
> > >
> > > For rbd, there is no such limit in QEMU except the size of the virtio
> > > ring buffer, but librbd may add limits of its own.
> > >
> > > For files, there's no limit if you use aio=threads, but the more I/O
> > > operations you trigger the more threads QEMU will create.  After 10
> > > seconds of idle the threads will be destroyed.
> > >
> > > Also for files, the limit is 128 per disk if you use aio=native.  You
> > > can only change it by recompilation only.
> >
> > Has anyone looked at enforcing some limits?  I'm okay with throttling
> > performance if necessary, but I really don't want to trigger the
> > OOM-killer.
>
> I forgot about "-drive ...,iops_max=NNN". :)

I'm not sure it's actually useful though, since it specifies the max IO 
operations per second, not the maximum number of in-flight operations.

If the far end can't keep up with the requested rate of operations, then 
the number of in-flight operations is going to grow even if you throttle 
the number of operations per second.

Chris