Re: [Qemu-devel] is there a limit on the number of in-flight I/O operations?

[Chris Friesen]

On 07/21/2014 01:47 PM, Benoît Canet wrote:
> The Monday 21 Jul 2014 à 09:35:29 (-0600), Chris Friesen wrote :
> > On 07/21/2014 09:15 AM, Benoît Canet wrote:
> > > The Monday 21 Jul 2014 à 08:59:45 (-0600), Chris Friesen wrote :
> > > > On 07/19/2014 02:45 AM, Benoît Canet wrote:
> > > >
> > > > > I think in the throttling case the number of in flight operation is limited by
> > > > > the emulated hardware queue. Else request would pile up and throttling would be
> > > > > inefective.
> > > > >
> > > > > So this number should be around: #define VIRTIO_PCI_QUEUE_MAX 64 or something 
> > > > > like than that.
> > > >
> > > > Okay, that makes sense.  Do you know how much data can be written as part of
> > > > a single operation?  We're using 2MB hugepages for the guest memory, and we
> > > > saw the qemu RSS numbers jump from 25-30MB during normal operation up to
> > > > 120-180MB when running dbench.  I'd like to know what the worst-case would
> > > > be.
>
> At first start QEMU start only a bunch of IO threads.
> When IOs activity kick some other IO threads will be started and can show up in
> memory measurement because the same physical memory will be referenced by 
> multiple threads.
>
> Are you sure this is not the case you are seeing ?
>
> The H option of ps on the host could help.

I'm pretty sure that this wouldn't change the RES (aka RSS) value, since 
all those threads are sharing the same address space.

If we create IO threads on activity though, we could end up causing some 
overhead due to the per-thread stack (8MB by default).  Do we have a 
limit on how many IO threads could get created?

Chris