Re: [Qemu-devel] [RFC]QEMU disk I/O limits

[Vivek Goyal]

On Tue, May 31, 2011 at 03:19:56PM +0100, Daniel P. Berrange wrote:
> On Tue, May 31, 2011 at 10:10:37AM -0400, Vivek Goyal wrote:
> > On Tue, May 31, 2011 at 02:56:46PM +0100, Daniel P. Berrange wrote:
> > > On Tue, May 31, 2011 at 09:45:37AM -0400, Vivek Goyal wrote:
> > > > On Mon, May 30, 2011 at 01:09:23PM +0800, Zhi Yong Wu wrote:
> > > > > Hello, all,
> > > > > 
> > > > >     I have prepared to work on a feature called "Disk I/O limits" for 
> > > > > qemu-kvm projeect.
> > > > >     This feature will enable the user to cap disk I/O amount 
> > > > > performed by a VM.It is important for some storage resources to be 
> > > > > shared among multi-VMs. As you've known, if some of VMs are doing 
> > > > > excessive disk I/O, they will hurt the performance of other VMs.
> > > > > 
> > > > 
> > > > Hi Zhiyong,
> > > > 
> > > > Why not use kernel blkio controller for this and why reinvent the wheel
> > > > and implement the feature again in qemu?
> > > 
> > > The finest level of granularity offered by cgroups apply limits per QEMU
> > > process. So the blkio controller can't be used to apply controls directly
> > > to individual disks used by QEMU, only the VM as a whole.
> > 
> > So are multiple VMs using same disk. Then put multiple VMs in same
> > cgroup and apply the limit on that disk.
> > 
> > Or if you want to put a system wide limit on a disk, then put all
> > VMs in root cgroup and put limit on root cgroups.
> > 
> > I fail to understand what's the exact requirement here. I thought
> > the biggest use case was isolation one VM from other which might
> > be sharing same device. Hence we were interested in putting 
> > per VM limit on disk and not a system wide limit on disk (independent
> > of VM).
> 
> No, it isn't about putting limits on a disk independant of a VM. It is
> about one VM having multiple disks, and wanting to set different policies
> for each of its virtual disks. eg
> 
>   qemu-kvm -drive file=/dev/sda1 -drive file=/dev/sdb3
> 
> and wanting to say that sda1 is limited to 10 MB/s, while sdb3 is
> limited to 50 MB/s.  You can't do that kind of thing with cgroups,
> because it can only control the entire process, not individual
> resources within the process.

With IO controller you can do that. Limits are "per cgroup per disk".
So once you have put a VM in a cgroup, you can specify two differnt
limits for two disk for that cgroup.

There are 4 relevant files per cgroup.

blkio.throttle.read_bps_device
blkio.throttle.write_bps_device
blkio.throttle.read_iops_device
blkio.throttle.write_iops_device

And syntax of these files is.

device_major:device_minor  <rate_limit>

Ex.

8:16    1024000

This means from a specified cgroup, on disk with major:minor 8:16, don't
allow read BW higher than 1024000 bytes per second.

Thanks
Vivek