Re: [Qemu-devel] [PATCH 4/4] qemu-img: conditionally discard target on convert

[ronnie sahlberg]

On Thu, Jul 18, 2013 at 11:08 PM, Peter Lieven <address@hidden> wrote:
> On 19.07.2013 07:58, Paolo Bonzini wrote:
>>
>> Il 18/07/2013 21:28, Peter Lieven ha scritto:
>>>
>>> thanks for the details. I think to have optimal performance and best
>>> change for unmapping in qemu-img convert
>>> it might be best to export the OPTIMAL UNMAP GRANULARITY
>>
>> Agreed about this.
>>
>>> as well as the write_zeroes_w_discard capability via the BDI
>>
>> But why this?!?  It is _not_ needed.  All you need is to change the
>> default of the "-S" option to be the OPTIMAL UNMAP GRANULARITY if it is
>> nonzero.
>
> 2 reasons:
> a) Does this guarantee that the requests are aligned to multiple of the -S
> value?
>
> b) If I this flag exists qemu-img convert can do the "zeroing" a priori.
> This
> has the benefit that combined with bdrv_get_block_status requests before it
> might
> not need to touch large areas of the volume. Speaking for iSCSI its likely
> that
> the user sets a fresh volume as the destination, but its not guaranteed.
> With the Patch 4 there are only done a few get_block_status requests to
> verify
> this. If we just write zeroes with BDRV_MAY_UNMAP, we send hundreds or
> thousands of writesame requests for possibly already unmapped data.
>
> To give an example. If I take my storage with an 1TB volume. It takes about
> 10-12
> get_block_status requests to verify that it is completely unmapped. After
> this
> I am safe to set has_zero_init = 1 in qemu-img convert.
>
> If I would convert a 1TB image to this target where lets say 50% are at leat
> 15MB
> zero blocks (15MB is the OUG of my storage) it would take ~35000 write same
> requests to achieve the same.

I am not sure I am reading this right, but you dont have to writesame
exactly 1xOUG to get it to unmap.
nxOUG will work too,
So instead of sending one writesame for each OUG range, you can send
one writesame for every ~10G or so.
Say 10G is ~667 OUGs for your case, so you can send
writesame for ~667xOUG in each command and then it would "only" take
~100 writesames instead of ~35000.

So as long as you are sending in multiples of OUG you should be fine.


>
> Peter
>
>>
>> Paolo
>>
>>> and than zero
>>> out the whole device with bdrv_write_zeroes and the BDRV_MAY_DISCARD
>>> flag.
>>>
>>> the logic for this is already prepared in patch4 (topic of this email).
>>> except that
>>> i have to exchange bdrv_discard with bdrv_write_zeroes w/
>>> BDRV_MAY_DISCARD.
>>>
>>> what do you think?
>>>
>>>
>>>
>>>>
>>>> On Thu, Jul 18, 2013 at 11:43 AM, Peter Lieven <address@hidden> wrote:
>>>>>
>>>>> Am 18.07.2013 um 16:35 schrieb Paolo Bonzini <address@hidden>:
>>>>>
>>>>>> Il 18/07/2013 16:32, Peter Lieven ha scritto:
>>>>>>>>
>>>>>>>> (Mis)alignment and granularity can be handled later.  We can ignore
>>>>>>>> them
>>>>>>>> for now.  Later, if we decide the best way to support them is a
>>>>>>>> flag,
>>>>>>>> we'll add it.  Let's not put the cart before the horse.
>>>>>>>>
>>>>>>>> BTW, I expect alignment!=0 to be really, really rare.
>>>>>>>
>>>>>>> To explain my concerns:
>>>>>>>
>>>>>>> I know that my target has internal page size of 15MB. I will check
>>>>>>> what
>>>>>>> happens
>>>>>>> if I deallocate this 15MB in chunks of lets say 1MB. If the page gets
>>>>>>> unprovisioned
>>>>>>> after the last chunk is unmapped it would be fine :-)
>>>>>>
>>>>>> You're talking of granularity here, not (mis)alignment.
>>>>>
>>>>> you are right. for the target i am talking about this is 30720 512-byte
>>>>> blocks for the granularity (pagesize) and 0 for the alignment.
>>>>> i will see what happens if I write same w/unmap the whole 30720 blocks
>>>>> in smaller blocks ;-) otherwise i will have to add support for honoring 
>>>>> this
>>>>> values in qemu-img convert
>>>>> as a follow up.
>>>>>
>>>>> Peter
>>>>>
>>>
>>>
>
>
> --
>
> Mit freundlichen Grüßen
>
> Peter Lieven
>
> ...........................................................
>
>   KAMP Netzwerkdienste GmbH
>   Vestische Str. 89-91 | 46117 Oberhausen
>   Tel: +49 (0) 208.89 402-50 | Fax: +49 (0) 208.89 402-40
>   address@hidden | http://www.kamp.de
>
>   Geschäftsführer: Heiner Lante | Michael Lante
>   Amtsgericht Duisburg | HRB Nr. 12154
>   USt-Id-Nr.: DE 120607556
>
> ...........................................................
>