Re: [Qemu-devel] [PATCH] vmdk: improve streamOptimized vmdk support

[Milos Vyletel]

I will update description in v2 with more info.

On Wed, Jul 30, 2014 at 4:09 AM, Stefan Hajnoczi <address@hidden> wrote:
> On Tue, Jul 29, 2014 at 3:49 PM, Milos Vyletel <address@hidden> wrote:
>> On Tue, Jul 29, 2014 at 10:37 AM, Stefan Hajnoczi <address@hidden> wrote:
>>> On Tue, Jul 29, 2014 at 2:46 PM, Milos Vyletel <address@hidden> wrote:
>>>> On Tue, Jul 29, 2014 at 9:37 AM, Stefan Hajnoczi <address@hidden> wrote:
>>>>> On Mon, Jul 07, 2014 at 10:54:27AM -0400, Milos Vyletel wrote:
>>>>>> VMDK's streamOptimized format is different that regular sparse format.
>>>>>> L1(GD) and L2(GT) tables are not predefined but rather generated and
>>>>>> written during image creation mainly because there is no way to tell
>>>>>> how much space data will occupy once they are compressed. Also the
>>>>>> location of header, L1 and L2 tables differs.
>>>>>>
>>>>>> - L2 tables (grain tables) are written after all grains they point to
>>>>>> - L1 tables are written after all grains and L2 tables
>>>>>> - footer at the end is used instead of header in first sector
>>>>>>
>>>>>> This patch improves streamOptimized support and adds possibility to
>>>>>> create true streamOptimized images using qemu-img. Some of the changes
>>>>>> are from VMDK specs, some of them from hexdump-ing images from VMWare
>>>>>> and VirtualBox.
>>>>>>
>>>>>> I have compared these images to the ones generated by VMWare and vbox
>>>>>> and they are identical with the exception of DescriptorFile that has
>>>>>> some differences but none that would change behavior(CID and some
>>>>>> additional DDB entries differ) and streamOptimized image generated from
>>>>>> raw image was succesfully imported (as OVA) into VMWare ESXi and Oracle
>>>>>> OVM.
>>>>>>
>>>>>> Signed-off-by: Milos Vyletel <address@hidden>
>>>>>> ---
>>>>>>  block/vmdk.c |  363 
>>>>>> +++++++++++++++++++++++++++++++++++++++++++++-------------
>>>>>>  1 files changed, 281 insertions(+), 82 deletions(-)
>>>>>
>>>>> What does this patch do beyond what QEMU already supports today?
>>>>>
>>>>> Is there a particular application that rejected QEMU's streamOptimized
>>>>> images?  Is this a bug fix?
>>>>>
>>>>> Please use scripts/checkpatch.pl to check coding style.
>>>>>
>>>>> Fam: Please review
>>>>>
>>>>
>>>> Images created/converted using QEMU were rejected by VMWare ESXi,
>>>> vCloud, VirtualBox and Oracle OVM. I did not try anything else.
>>>>
>>>> Generally speaking streamOptimized format is not followed by QEMU.
>>>> Instead regular VMDK format + encryption is used. We can say this is
>>>> bug fix since streamOptimzed format is already included but does not
>>>> work well.
>>>>
>>>> I'll check the style and fix the code. I'll postpone resubmit until
>>>> further review is done since I'm sure there will be things that will
>>>> need to be fixed.
>>>
>>> What was the command-line you used?
>>>
>>> qemu-img convert -f qcow2 -O vmdk -o subformat=streamOptimized
>>> input.qcow2 output.vmdk
>>>
>>> Stefan
>>
>> I've been using test image from
>> http://wiki.qemu.org/download/linux-0.2.img.bz2 and converting it like
>> this:
>>
>> qemu-img convert -O vmdk -o
>> adapter_type=lsilogic,compat6,subformat=streamOptimized linux-0.2.img
>> linux-0.2.vmdk
>
> Okay, great.  I wanted to check that you used subformat=streamOptimized.
>
> If possible, please be more specific in the commit description about
> what caused ESXi and other software to reject the image and what
> exactly this patch changes.
>
> Stefan