Re: [Qemu-discuss] qemu-nbd or qcow2 or something else ?

[Pascal]

hello,

same problem with qcow format (with smaller files) :-(
on the other hand, when the two synchronization commands are uncommented
(see the second results), the problem disappears: it would probably be a
"cache" problem as mentioned by Max Reitz.

#!/bin/bash
set -euv
uname -a
modprobe nbd max_part=16
qemu-img -V
qemu-img create -f qcow /tmp/a.img 512M
qemu-img create -f qcow /tmp/b.img 512M
qemu-nbd -V
qemu-nbd -f qcow -c /dev/nbd0 /tmp/a.img
qemu-nbd -f qcow -c /dev/nbd1 /tmp/b.img
fdisk /dev/nbd0 <<<$'n\n\n\n\n\nt\n7\nw'
fdisk -l /dev/nbd0
mkfs.ntfs /dev/nbd0p1
ntfsfix -n /dev/nbd0p1
file -s /dev/nbd0p1
fdisk -l /dev/nbd1
# sync
# sudo sysctl -q vm.drop_caches=3
cat /dev/nbd0 > /dev/nbd1
blockdev --rereadpt /dev/nbd1
fdisk -l /dev/nbd1
ntfsfix -n /dev/nbd1p1 || true
file -s /dev/nbd1p1
sha1sum /dev/nbd0 /dev/nbd1
sha1sum /dev/nbd0p1 /dev/nbd1p1
qemu-nbd -d /dev/nbd0
qemu-nbd -d /dev/nbd1
rm -f /tmp/a.img
rm -f /tmp/b.img


8<----------------------------------


uname -a
Linux archlive 4.9.65-1-lts #1 SMP Fri Nov 24 13:02:16 CET 2017 x86_64
GNU/Linux

modprobe nbd max_part=16

qemu-img -V
qemu-img version 2.10.1
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

qemu-img create -f qcow /tmp/a.img 512M
Formatting '/tmp/a.img', fmt=qcow size=536870912

qemu-img create -f qcow /tmp/b.img 512M
Formatting '/tmp/b.img', fmt=qcow size=536870912

qemu-nbd -V
qemu-nbd 2.10.1
Written by Anthony Liguori.
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

qemu-nbd -f qcow -c /dev/nbd0 /tmp/a.img

qemu-nbd -f qcow -c /dev/nbd1 /tmp/b.img

fdisk /dev/nbd0 <<<$'n\n\n\n\n\nt\n7\nw'
...
Synchronisation des disques.

fdisk -l /dev/nbd0
Disque /dev/nbd0 : 512 MiB, 536870912 octets, 1048576 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x0a758e24
Périphérique Amorçage Début     Fin Secteurs Taille Id Type
/dev/nbd0p1            2048 1048575  1046528   511M  7 HPFS/NTFS/exFAT

mkfs.ntfs /dev/nbd0p1
The partition start sector was not specified for /dev/nbd0p1 and it could
not be obtained automatically.  It has been set to 0.
The number of sectors per track was not specified for /dev/nbd0p1 and it
could not be obtained automatically.  It has been set to 0.
The number of heads was not specified for /dev/nbd0p1 and it could not be
obtained automatically.  It has been set to 0.
Cluster size has been automatically set to 4096 bytes.
To boot from a device, Windows needs the 'partition start sector', the
'sectors per track' and the 'number of heads' to be set.
Windows will not be able to boot from this device.
Initializing device with zeroes: 100% - Done.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.

ntfsfix -n /dev/nbd0p1
Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/nbd0p1 was processed successfully.

file -s /dev/nbd0p1
/dev/nbd0p1: DOS/MBR boot sector, code offset 0x52+2, OEM-ID "NTFS    ",
sectors/cluster 8, Media descriptor 0xf8, sectors/track 0, dos < 4.0
BootSector (0x80), FAT (1Y bit by descriptor); NTFS, sectors 1046527, $MFT
start cluster 4, $MFTMirror start cluster 65407, bytes/RecordSegment
2^(-1*246), clusters/index block 1, serial number 0479f2c2b38afbfec

fdisk -l /dev/nbd1
Disque /dev/nbd1 : 512 MiB, 536870912 octets, 1048576 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets

# sync
# sudo sysctl -q vm.drop_caches=3

cat /dev/nbd0 > /dev/nbd1

blockdev --rereadpt /dev/nbd1

fdisk -l /dev/nbd1
Disque /dev/nbd1 : 512 MiB, 536870912 octets, 1048576 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x0a758e24
Périphérique Amorçage Début     Fin Secteurs Taille Id Type
/dev/nbd1p1            2048 1048575  1046528   511M  7 HPFS/NTFS/exFAT

ntfsfix -n /dev/nbd1p1 || true
Mounting volume... NTFS signature is missing.
FAILED
Attempting to correct errors... NTFS signature is missing.
FAILED
Failed to startup volume: Invalid argument
NTFS signature is missing.
Trying the alternate boot sector
Unrecoverable error
Volume is corrupt. You should run chkdsk.
No change made

file -s /dev/nbd1p1
/dev/nbd1p1: data

sha1sum /dev/nbd0 /dev/nbd1
995b2fca8aecc9e33030e55335acbb3cb0bae3a3  /dev/nbd0
995b2fca8aecc9e33030e55335acbb3cb0bae3a3  /dev/nbd1

sha1sum /dev/nbd0p1 /dev/nbd1p1
629bd3b9f5016d6b1f4d5ccefb3e253c1183650c  /dev/nbd0p1
85818bbde68386daae96767ef0b46a0bf906b2e5  /dev/nbd1p1

qemu-nbd -d /dev/nbd0
/dev/nbd0 disconnected

qemu-nbd -d /dev/nbd1
/dev/nbd1 disconnected

rm -f /tmp/a.img

rm -f /tmp/b.img


8<----------------------------------


*...( **the same results as above for the corresponding first part )...*

sync
sudo sysctl -q vm.drop_caches=3

cat /dev/nbd0 > /dev/nbd1

blockdev --rereadpt /dev/nbd1

fdisk -l /dev/nbd1
Disque /dev/nbd1 : 512 MiB, 536870912 octets, 1048576 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x2bc8ee5c
Périphérique Amorçage Début     Fin Secteurs Taille Id Type
/dev/nbd1p1            2048 1048575  1046528   511M  7 HPFS/NTFS/exFAT

ntfsfix -n /dev/nbd1p1 || true
Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/nbd1p1 was processed successfully.

file -s /dev/nbd1p1
/dev/nbd1p1: DOS/MBR boot sector, code offset 0x52+2, OEM-ID "NTFS    ",
sectors/cluster 8, Media descriptor 0xf8, sectors/track 0, dos < 4.0
BootSector (0x80), FAT (1Y bit by descriptor); NTFS, sectors 1046527, $MFT
start cluster 4, $MFTMirror start cluster 65407, bytes/RecordSegment
2^(-1*246), clusters/index block 1, serial number 0405697661d6d540a

sha1sum /dev/nbd0 /dev/nbd1
856a6671e9737c9be63e1458dd4b2a3e07027ae7  /dev/nbd0
856a6671e9737c9be63e1458dd4b2a3e07027ae7  /dev/nbd1

sha1sum /dev/nbd0p1 /dev/nbd1p1
bda6259d042e482131d5880305172e6f54c8c201  /dev/nbd0p1
bda6259d042e482131d5880305172e6f54c8c201  /dev/nbd1p1

qemu-nbd -d /dev/nbd0
/dev/nbd0 disconnected

qemu-nbd -d /dev/nbd1
/dev/nbd1 disconnected

rm -f /tmp/a.img

rm -f /tmp/b.img


2017-12-05 19:29 GMT+01:00 Programmingkid <address@hidden>:

> >
> > Message: 1
> > Date: Mon, 4 Dec 2017 19:04:36 +0100
> > From: Max Reitz <address@hidden>
> > To: Pascal <address@hidden>, address@hidden,
> >       address@hidden
> > Subject: Re: [Qemu-discuss] qemu-nbd or qcow2 or something else ?
> > Message-ID: <address@hidden>
> > Content-Type: text/plain; charset="utf-8"
> >
> > On 2017-12-01 18:56, Pascal wrote:
> >> hello,
> >>
> >> while doing some tests on the ntfs file system, I met some strange
> things
> >> with my qcow2 disk images.
> >> the images are on a partition mounted in tmpfs, but the result is the
> same
> >> when they are recorded on a partition in ext4 format.
> >> I don't know where the problem comes from : qemu-nbd or format qcow2 or
> >> something else ?
> >> do not hesitate if you want more informations.
> >
> > I see the issue here as well (and with raw, too).
> >
> > tl;dr: Seems like a kernel issue to me (CC-ing the NBD list because
> > that's the best I can do).
> >
> > When tracing the accesses, it appears that at least the NTFS header is
> > not read from the source disk when copying the data over.  I would guess
> > this is due to caching, because Linux has read that sector before the
> > mkfs.ntfs (so it was zero then).
> >
> > And the issue disappears if I insert a "blockdev --flushbufs /dev/nbd0"
> > after the mkfs.ntfs -- but not if I flush nbd0p1, interestingly.
> >
> > I would guess the kernel has different caches for the whole device and
> > each partition?  Well, that's nice.  Not sure if that is a bug or
> > whether that is just how it is...
> >
> > (But nbd just uses the normal blockdev partitioning, so I guess it's by
> > design?  (And the same issue appears with kpartx, too))
> >
> > You can also see this on the source volume alone:
> >
> > $ qemu-img create [...]
> > # qemu-nbd -c /dev/nbd0 [...]
> > # fdisk /dev/nbd0
> > # mkfs.ntfs /dev/nbd0p1
> >
> > # hexdump -C /dev/nbd0
> > [...]
> > 00000200  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> > |................|
> > *
> > 00102000  ff ff 00 07 00 00 00 00  00 00 00 00 00 00 00 00
> > |................|[...]
> >
> > # blockdev --flushbufs /dev/nbd0
> >
> > # hexdump -C /dev/nbd0
> > [...]
> > 00100000  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS
> > .....|
> > 00100010  00 00 00 00 00 f8 00 00  00 00 00 00 00 00 00 00
> > |................|
> > [...]
> >
> >
> > However, there is still an open question: I can't reproduce this with
> > loop or real devices.  I only see this with NBD.  Why?  After having dug
> > for too long into the kernel sources, my best guess right now is that
> > the kernel NBD driver might be missing some necessary flushes.  Whenever
> > one NBD device is accessed (through a partition or not), it is necessary
> > to flush all device nodes that are associated with it -- but loop
> > doesn't seem to be doing this, and I would expect the general partition
> > framework to handle this already.  Therefore, my best guess is a bad
> guess.
> >
> > But note that I can reproduce the issue with nbd-server and nbd-client
> > just fine:
> >
> > # dd if=/dev/zero of=/tmp/foo.img bs=1M count=2048
> > 2048+0 records in
> > 2048+0 records out
> > 2147483648 bytes (2.1 GB, 2.0 GiB) copied, 0.517711 s, 4.1 GB/s
> >
> > # nbd-server 10809 /tmp/foo.img
> > ** (process:17331): WARNING **: Specifying an export on the command line
> > no longer uses the oldstyle protocol.
> >
> > # nbd-client localhost /dev/nbd0
> > Warning: the oldstyle protocol is no longer supported.
> > This method now uses the newstyle protocol with a default export
> > Negotiation: ..size = 2048MB
> > bs=1024, sz=2147483648 bytes
> >
> > # echo -e 'n\n\n\n\n\nt\n7\nw' | fdisk /dev/nbd0
> > [...]
> >
> > # mkfs.ntfs /dev/nbd0p1
> > [...]
> >
> > # hexdump -C /dev/nbd0 | less
> > [...]
> > 00000200  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> > |................|
> > *
> > 00102000  ff ff 00 07 00 00 00 00  00 00 00 00 00 00 00 00
> > |................|[...]
> >
> > # blockdev --flushbufs /dev/nbd0
> >
> > # hexdump -C /dev/nbd0 | less
> > [...]
> > 00100000  eb 52 90 4e 54 46 53 20  20 20 20 00 04 04 00 00  |.R.NTFS
> > .....|
> > 00100010  00 00 00 00 00 f8 00 00  00 00 00 00 00 00 00 00
> > |................|[...]
> >
> > Max
>
> I was just wondering if you have tried the qcow format yet. I was having
> issues with my qcow2 image file becoming corrupted to the point I couldn't
> boot Windows from it. When I switched over to qcow, the problem went away.
>
>