Re: [Qemu-devel] [PATCH for 1.4] block/vpc: Fix size calculation

[Jeff Cody]

On Mon, Feb 11, 2013 at 10:33:53PM +0100, Stefan Weil wrote:
> Am 11.02.2013 20:38, schrieb Jeff Cody:
> > I've done some testing back and forth between VPC, Win Server 2012
> > Hyper-V, and qemu/kvm with and without this patch applied.
> >
> > The short: this patch will break Virtual PC VHD compatibility (as it
> > exists under Win7), but allow compatibility for VHD files created by
> > Hyper-V (at least under Server 2012).
> >
> > VPC and Server 2012 Hyper-V are also incompatible with each other,
> > when it comes to VHD files, if that is any consolation.
> >
> > I used sha1sums between the different combination of VHD files filled to
> > capacity with random data under VPC, qemu/kvm (with & without the
> > patch), and Hyper-V.  Without this patch we are an exact math with
> > VPC, and with this patch we are an exact match with Hyper-V.
> >
> > However, perhaps we can support both, depending on who created the VHD
> > file.  There is a "Creator Application" field in the VHD header - for
> > the VPC-created VHD file, that field is ASCII "vpc ".  For the Hyper-V
> > created VHD file, that field is ASCII "win ".  If we see the field as
> > "vpc ", we could default to using the CHS geometry found in the
> > header - otherwise, do as this patch does, and go by the virtual disk
> > size.
> >
> > Jeff
> 
> Hi Jeff,
> 
> could you please give some more details of your VPC test scenario?

Hello Stefan,

Sure, no problem.

I will answer each of your questions inline below, but before that
perhaps an overview would be useful:

Ultimately, it doesn't matter if VPC or Hyper-V created the image,
what matters is who fills the data in, or partitions those images.
Consider the two cases below:

A.) Virtual PC appears to go by the CHS data in the header/footer, as
qemu currently does without the patch.  

B.) Hyper-V, and qemu with the patch, use the virtual disk size. This
usually yields a disk size slightly larger than what is seen in A).

If you are importing from A->B, then there _likely_ will not be any
issues, but of course this cannot be 100% guaranteed.

If you are importing from B->A then there _will_ likely be issues, as
you are going from a larger drive to smaller drive.

This patch just changes which one is more supported from a VHD
perspective - Virtual PC, or Hyper-V.  If this was fresh code without
existing support, I would say being compatible with Hyper-V would be
more important. 

However, there are two important factors:

1.) QEMU currently is able to import and export dynamic VHD files
from/to Virtual PC.  This would break that..

2.) I would wager most use cases of Hyper-V would be using VHDX,
instead of VHD anyway.  I am working currently on implementing VHDX
support to address see (my work in progress is over at github: 
https://github.com/codyprime/qemu-kvm-jtc.git , branch jtc-vhdx-work).

3.) With some more effort, we can reasonably support both use cases,
as long as the user has a way to force method if needed.
> 
> How did you create the VHD disk imagesfor VPC?

I created the images with Virtual PC under Windows 7.  I created a new
virtual machine, with a new drive image.  I booted from a live ISO to
partition the image, and fill it with data. I created images of
varying sizes, but all of the same type: dynamic vhd images.

The easiest way to replicate the problem is to use a smaller image,
because you can fill it up with data without much trouble. For my
small image test, I used ~130MB image.

I also created images under Hyper-V, of the same type.  As you know,
those images work fine with the patch.  And again, it is not so much
the creator, but who partitioned or filled the drive with data.

One easy way to reproduce the discrepancy:

Using the same image, from either VPC or Hyper-V, inside the guest
fill the test image with pseudo-random data, and make note of the
sha1sum:

e.g., from VPC, inside a linux guest: 
    "cat /dev/urandom > /dev/sdX; sha1sum /dev/sdX"

Then, under either Hyper-V or qemu with your patch, verify that sum:
    "sha1sum /dev/vdX"  (qemu w/virtio)
    "sha1sum /dev/sdX"  (Hyper-V)

You'll notice that with your patch, the sha1sum will not match the one
computed from the guest with VPC as the host.  However, if you try
qemu _without_ your patch, the sha1sum will match.  Both Hyper-V and
qemu will compute the same sha1sum, however.

You can then do this in reverse, with the same image under Hyper-V:
fill it with pseudo-random data, make note of the sha1sum, and check
that sha1sum from VPC and qemu.  You'll notice the same - with your
patch, qemu and Hyper-V agree on sha1sums, and VPC does not.  Without
your patch, qemu and VPC agree on sha1sums, and Hyper-V does not.

> Which guest OS did you run?

I used Linux, either Fedora 18 or a lightweight live ISO of puppy
Linux (slacko 5.4).  Both gave the same results.  The version of Linux
did not seem to matter (as is to be expected).

> Which disk size does the guest OS report?

The guest OS under Virtual PC reports a smaller size, as observed by
fdisk.  For instance, on my smaller image, the disk size under Virtual
PC was reported as "134111232 bytes".  Under Hyper-V and qemu/kvm with
your patch, that same image is reported as "134217728 bytes".  Without
your patch, qemu again agrees with VPC, with "134111232 bytes".

> Does VPC behave differently when it gets disk images with
> a different signature (for example from Hyper V)?

Yes, and this is the case most likely to have an immediately noticeable
failure.  Linux refuses to mount the internal partition, because of
the incorrect geometry and too-large partition.

> 
> I'd like to reproduce your test results in my environment.
> 
> Does the patch really break Virtual PC VHD compatibility?
> 

Most likely, with this patch import of a VHD will in most cases be
fine, because we will just detect a slightly larger disk in qemu.

However, if the guest modifies the image, we don't know that it won't
do so in a way that will case the image to not be readable by Virtual
PC.  Most likely, the operation that would cause this would be
something like repartitioning the image - but anything that makes use
of the slightly large space would naturally lose that data going to
Virtual PC.

> Or does VPC simply pass the size calculated from CHSto the
> guest - no matter how the disk image was created?In this case
> I see no need that QEMU must show the same behaviour.
> It's important to implement the VHD specification correctly,
> but we should not duplicate bugs of other emulators just
> to be compatible with them.
> 

I disagree. The purpose of supporting VHD is to be compatible with
other emulators, so to the extent possible that we can do that, we
should. 

> Thanks,
> 
> Stefan
> 

Thanks,
Jeff