Re: [Qemu-devel] qemu-img convert from raw to vmdk does not work in vmware esxi

[Milos Vyletel]

On Mon, Jun 16, 2014 at 2:36 AM, Fam Zheng <address@hidden> wrote:
> On Fri, 06/13 15:00, Milos Vyletel wrote:
>> Hi,
>>
>> I've tried to convert my VM image from raw format to vmdk to create
>> OVF/OVA archive so that we can deploy our OS on other hypervisors. the
>> problem was that no matter how I've converted to vmdk vmware ESXi
>> (tried 4.1 and 5.5) complained that it was: "Not a supported disk
>> format (sparse VMDK too old)".
>>
>> I've found similar report on this list from 2011 but it was never solved.
>> http://lists.gnu.org/archive/html/qemu-devel/2011-10/msg02463.html
>>
>> Anyway, I've managed to find the problem. Even though vmdk specs
>> (latest I've found were at
>> https://www.vmware.com/support/developer/vddk/vmdk_50_technote.pdf?src=vmdk)
>> say that version of the vmdk is either 1 or 2 vmware actually uses
>> version 3 for streamOptimized vmdk format. When I've patched qemu-img
>> to set version to 3 for streamOptimzed (or more specifically
>> compressed) vmdk format import worked just fine.
>>
>> These are the options I've use to convert
>> qemu-img version 2.0.50, Copyright (c) 2004-2008 Fabrice Bellard
>> qemu-img convert -p -f raw /storage/gs.img -O vmdk -o
>> adapter_type=lsilogic,subformat=streamOptimized,compat6
>> /storage/exp/gs/gs.vmdk
>>
>> Below is quick and dirty patch I've come up with that does the trick.
>> I did not spend too much time looking into code to be 100% sure it's
>> correct so any comments are welcome.
>>
>> Milos
>>
>> ---
>> diff --git a/block/vmdk.c b/block/vmdk.c
>> index b8a4762..71d53b8 100644
>> --- a/block/vmdk.c
>> +++ b/block/vmdk.c
>> @@ -645,7 +645,9 @@ static int vmdk_open_vmdk4(BlockDriverState *bs,
>>          error_set(errp, QERR_UNKNOWN_BLOCK_FORMAT_FEATURE,
>>                    bs->device_name, "vmdk", buf);
>>          return -ENOTSUP;
>> -    } else if (le32_to_cpu(header.version) == 3 && (flags & BDRV_O_RDWR)) {
>> +    } else if (le32_to_cpu(header.version) == 3 &&
>> +               (flags & BDRV_O_RDWR) &&
>> +               (flags & VMDK4_FLAG_COMPRESS)) {
>
> I think this should be:
>
>     } else if (le32_to_cpu(header.version) == 3
>                && (flags & bdrv_o_rdwr)
>                && !(le64_to_cpu(header.flags) & vmdk4_flag_compress)) {
>
> We should look in header, because flags is not containing the bit we want.
>
> Fam
>
>>          /* VMware KB 2064959 explains that version 3 added support for
>>           * persistent changed block tracking (CBT), and backup software can
>>           * read it as version=1 if it doesn't care about the changed area
>> @@ -1562,7 +1564,7 @@ static int vmdk_create_extent(const char
>> *filename, int64_t filesize,
>>      }
>>      magic = cpu_to_be32(VMDK4_MAGIC);
>>      memset(&header, 0, sizeof(header));
>> -    header.version = zeroed_grain ? 2 : 1;
>> +    header.version = zeroed_grain ? 2 : (compress ? 3 : 1);
>>      header.flags = VMDK4_FLAG_RGD | VMDK4_FLAG_NL_DETECT
>>                     | (compress ? VMDK4_FLAG_COMPRESS | VMDK4_FLAG_MARKER : 
>> 0)
>>                     | (zeroed_grain ? VMDK4_FLAG_ZERO_GRAIN : 0);
>>

Thanks, that makes more sense. My change was obviously wrong.

Anyway, I'll test this change today. I've hit some core dumps on
latest qemu that I'll send in separate mail.

One more thing I've noticed is that VMWare does not like the version 1
on non-compressed images too. I've tried streamOptimized and
monolithicSparse and both of them failed to work in ESXi unless
version number was 3. The fact is that monolithicSparse was declined
even with version 3 but because it was "uncompressed disk".

Same goes for zeroed_grain which should be version 2 per specs but
still does not work unless it's version 3. It almost seems like there
is a check on vmware side that errors out if version < 3...

Milos