Re: [Qemu-devel] High memory usage by multiqueue

[Jason Wang]

On 02/20/2013 02:59 AM, Edivaldo de Araujo Pereira wrote:
> Dear Jason,
>
> In the last 2 weeks, I noticed a significant increase in the memory usage of 
> qemu. Just to assess the stability of my system, I usually run a kind of 
> stress test, every time I install a new kernel in my host or in a custom VM 
> that I use in my classes, or when I compile a new snapshot of qemu. 
>
> The test is quite simple, but somewhat brutal: 200 qemu guests (1 cpu, 256MB, 
> no hd) running on a single host (4 core, 8 threads, 8GB). It's obvious that 
> such thing doesn't serve any purpose other than to say it runs... 80% of all 
> memory and 35% of all processing just to make 200 guests little more than 
> idle... But the point is, when everything is ok, it runs very well, and I can 
> even put some 240Mbits of traffic in the virtual net.
>
> In my tests, each VM uses ~180MB (thanks to ksm, 200*180MB<8GB); but in 
> recent weeks, the memory footprint of the MVs jumped to ~230MB, along whith 
> some change in its behaviour that rends completely broken the magic of ksm 
> (at least in my test); in other words, I can no more run 200 guests in my 
> machine.
>
> So I did a "git bisect", as Stefan told some time ago, and I found the 
> problem is related to multiqueue support. More precisely, the constant 
> MAX_QUEUE_NUM 1024 in include/net/net.h.
>
> As long as I understand, such a high number of queues is at least a little 
> bit out of reality... Wouldn't it imply having possibly that number of 
> threads runing network traffic? Problem is, I can not imagine the total 
> number of logical processors a guest would need to have, just to take 
> advantage of such a number of network traffic thhreads. By the way, that 
> number is many times higher than supported by any physical network adapter I 
> have read about.
>
> So I've tampered a little with that number, and found that setting it to 4 or 
> 8 reduces drastically the memory usage, without disrupting any network 
> function (in my test...); yet this is twice the numbers of some examples I've 
> seen in wmware related texts. The good news: my stress test runs smoothly 
> again... well, almost, for I'm not using multiqueue yet...
>
> So, I think it would be good to find a better approach; dynamic allocation 
> would be nice, in accordance with command line parameters, wouldn't it?

Yes, it's a bug. The dynamic allocation is better(1.5). For 1.4, I think
we can simply limit the number to 8. Will post the patch.

Thanks
>
> Thank you, very much
>
> Edivaldo de Araújo Pereira
>