Re: [Qemu-devel] [PATCH 0/3] info blockstats (block-qcow2): show highest allocated offset (bytes)

[Shahar Frank]

Uri Lublin wrote:
> Anthony Liguori wrote:
> > Uri Lublin wrote:
> > > From: Uri Lublin <address@hidden>
> > >
> > > This patchset let the user know the highest allocated byte of qcow2 
> > > images.
> > > Actually it's the first unallocated byte after the highest byte written,
> > > cluster-size aligned.
> > >
> > > The highest allocated byte gives a maximal limit (easy to calculate)
> > > to the number of bytes allocated for that image, and may hint how 
> > > many more allocations can be done before we reach end-of-file (end of 
> > > host block device).
> > > Although there may be many free blocks below that number (allocated 
> > > and freed)
> > > the file system can not deallocate those blocks, and they have to be 
> > > reused
> > > by qemu. Also note that due to fragmentation those free blocks may not
> > > be used on next allocations.
> > >
> > > It can be useful for truncation of backing file images (ftruncate).
> > > Also it may be useful for defragmentation later (although we'll need
> > > the number of free blocks as well).
> >
> > I'm having trouble seeing the utility of this as it seems to be not 
> > really reliable.  Surely, after a lot of work, you'll have one block 
> > far at the end of the file, no?  I don't see how knowing this location 
> > helps practically speaking.  Can you explain a little more about what 
> > you want to use this functionality for?
>
> Currently, qcow2 images can only grow, never shrink.
> The main usage would be to trigger an appropriate operation when a 
> threshold is reached. The threshold and operation are defined by a 
> management application.
> Basically we can do one of the following:
> 1. Defragment the qcow2 image (simplest way is to qemu-img convert it, 
> the best is to do it online if possible).
> 2. Allocate more space (especially when using LVM)
>
> I plan on adding another "blockstat" that shows the number of free 
> bytes/blocks/clusters for a qcow2 image. This would make it easier to 
> choose the appropriate operation above.

As Uri wrote this patch is part of a patch set that will include also 
free clusters statistics. Together these new statistics will enable an 
image repository system to quickly estimate how "fragmented" is a given 
image, so it can decide if to perform expensive 
cleaning/shrinking/defragmenting processes on that image.

This highest allocated byte is also critical to let you run qcow2 images 
over raw devices- it give you a good estimate how much space you still 
have for the image to expand. This is a bit pessimistic statistic, but 
due the much too simplistic allocation mechanism within qcow2 and the 
recent multiple clusters allocations optimizations, this may be also a 
pretty realistic figure.
Note that if you run qcow2 over expandable volumes such as LVM volumes, 
you can use this figure by monitoring process that will expand the 
volume once a threshold is passed.

Shahar
> Thanks,
>     Uri.