[Qemu-devel] GSoC - Improved VHD(Virtual Hard Disk) image format compatibility and usability

[Lyu Mitnick]

Hello all,

This is my only proposal for the Google Summer of Code. Since the student 

application deadline is very near, I have already submitted it to melange. 

Comments and suggestions are really appreciated.

--

Contact Information

 

Email : address@hidden

Phone Number : + 886-963233988

 

Biography

 

I am a 22 years old computer science student in National Taiwan University. With the age of 20, I have finished my undergraduate project “VNC (Virtual Network Computing) on EFI (Extensible Firmware Interface)”. We can remote control the other computer without entering OS. As far as I know, there is no one do the project I did. So I have met many problems and there is no relevant information on web. I use background knowledge about BIOS and System Software to solve these issues by myself. I learned not only EFI skill but also the ability to find the solution of problems during the project.

 

Why QEMU

 

I use QEMU to emulate various CPU in my embedded system class. I appreciate to QEMU and I want to contribute to it. On the other hand, I like techniques about system level software and I am so obsessed with the principle and implementation of virtualization and QEMU.

 

Why This Project

 

There isn’t fully support of VHD (Virtual Hard Disk) format in QEMU. I list the new features that are not yet supported ([1], [2], [3]) in QEMU in the following:

 

[1] Add “Fixed hard disk image” support into VHD format

[2] Add “Differencing hard disk image” support into VHD format

 

There are three image types of VHD (Virtual Hard Disk) format:

·       Fixed hard disk image

·       Dynamic hard disk image

·       Differencing hard disk image

However there is only “Dynamic hard disk image” support in QEMU.

 

[3] Add “Splitting hard disk images” support into VHD format

 

Some file systems, such as the FAT32 file system, have a 4-GB limit on file size. If the hard disk image expands more than 4 GB, hypervisor will split the hard disk image into another file, which QEMU doesn’t support yet.

 

[4] Add “asynchronous I/O” support into VHD format using coroutines

 

All newer block drivers support asynchronous reads and writes, only some old drivers (ie. block dirver for VHD) don’t. Adding asynchronous I/O to the block driver for VHD would improve its usability.

 

There are two implementation methods of asynchronous I/O, callbacks and coroutines. Coroutines is used to replace callbacks to avoid:

·       Splitting up code into separate functions across each blocking point

·       Lot of code end up packing/unpacking variables that need to passed to callbacks

·       Hard to follow the code

I would try to make use of the ongoing coroutine effort in this project.

 

I have an experience of porting binutils so I am familiar with object file/executable formats. I found there is a little relevance between object file/executable formats and virtual hard disk file format. So I choose this project as my first contribution to QEMU.

 

Schedule

 

4/9   – 4/25 Get familiar with Git, build system of QEMU, QEMU coding style and      

                   QEMU coding guideline

4/26 – 5/9   Add “Fixed hard disk image” support into VHD format

5/10 – 6/11 Add “Differencing hard disk image” support into VHD format

6/12 – 7/11 Add “Splitting hard disk images” support into VHD format

7/12 – 7/15 Prepare for midterm evaluation

7/16 – 8/15 Add “asynchronous I/O” support into VHD format using coroutine

8/16 – 8/22 Write document, testing

8/23 – 8/26 Prepare for final evaluation

 

Approach

 

The followings are the detailed description of the approach I will take. For each subproject, I list the basic steps to solve the problem and the modification points what I learned.

 

[1] Add “Fixed hard disk image” support into VHD format

 

“Fixed hard disk image” is “raw file with a footer” in QEMU languages.

 

Basic steps to solve the problem:

·       Study the specification of VHD

·       Trace the code of block/vpc.c

o   block/vpc.c implements block driver for VHD format

·       Take a look of block/bochs.c

o   block/bochs.c implements block driver for the “various” disk images used by Bochs

 

Modification points:

·       block_int.h:BLOCK_FLAG_FIXED

o   #define BLOCK_FLAG_FIXED    8

·       block_int.h:BLOCK_OPT_FIXED

o   #define BLOCK_OPT_FIXED    “fixed”, used as parameter options of vpc_create()

·       block/vpc.c:vpc_open()

o   No dyndisk_header

·       block/vpc.c:get_sector_offset()

o   Different formula (No modification in block/vpc.c:vpc_read())

·       block/vpc.c:vpc_create()

o   Deal with option BLOCK_OPT_FIXED

o   No dyndisk_header

o   No BAT (Block Allocation Table)

·       block/vpc.c:vpc_close()

o   No pagetable

·       block/vpc.c:vpc_create_options[]

o   Add element
{
    .name = BLOCK_OPT_FIXED,
    .type = OPT_FLAG,
    .help = “VHD fixed hard disk image”
}

 

[2] Add “Differencing hard disk image” support into VHD format

 

“Differencing hard disk image” is “VHD with backing file” in QEMU languages.

 

Basic steps to solve the problem:

·       Study the specification of VHD

·       Trace the code of block/vpc.c

·       Take a look of block/bochs.c

·       Take a look of block/qcow2.c

o   block/qcow2.c implements block driver for the QCOW version 2 format, which have backing file supported.

 

Modification points:

·       block/vpc.c:vpc_open()

o   Convert Parent Locator Entries to native path and store at backing_file

·       block/vpc.c:vpc_read()

o   Read the sectors marked dirty from the differencing hard disk and the sectors marked clean from the parent hard disk

o   Open the parent hard disk file on demand (Optional optimization)

·       block/vpc.c:vpc_create()

o   Deal with option BLOCK_OPT_BACKING_FMT

o   Set Disk Type, Unique ID, Parent Unique ID, Parent Time Stamp, Parent Unicode Name, Parent Locator Entries

·       block/vpc.c:vpc_close()

o   Close all opened files

·       block/vpc.c:vpc_create_options[]

o   Add element
{        
    .name = BLOCK_OPT_BACKING_FILE,        
    .type = OPT_STRING,        
    .help = "File name of a parent hard disk file"
}

 

[3] Add “Splitting hard disk images” support into VHD format

 

Basic steps to solve the problem:

·       Study the specification of VHD

·       Trace the code of block/vpc.c

o   This feature is not tied to the host platform or file system but image format

 

Modification points:

·       block/fat32.c

o   Design a block driver for fat32, which handles underlying 4-GB limit on file size and abstract a sequential continuous file.

·       block/vpc.c:vpc_create()

o   Assign fat32 block driver to bs->drv if opened file is the first file of split files and initialize the fat32 block driver.

o   Assign fat32 block driver to bs->drv if file is opened under FAT32 file system and initialize the fat32 block driver.

 

[4] Add “asynchronous I/O” support into VHD format using coroutine

 

Basic steps to solve the problem:

·       Trace the code of block/vpc.c

·       Take a look of block/qcow2.c (master tree)

·       Take a look of block/qcow2.c (Kevin’s branch)

 

Modification points:

·       block/vpc.c:vpc_aio_cancel()

·       block/vpc.c:vpc_aio_pool

·       block/vpc.c:vpc_aio_setup()

·       block/vpc.c:vpc_aio_readv()

·       block/vpc.c:vpc_aio_writev()

·       block/vpc.c:vpc_aio_flush()

 --

 

Thanks a lot

Mitnick