[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd
From: |
Jaze Lee |
Subject: |
Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd |
Date: |
Wed, 22 Feb 2017 17:05:22 +0800 |
I test on cache=none with sata disk, because i do not have ssd in my pc,
and I do not find any performance improved.
I do not know why the performance is not be improved....
The test case is:
boot vm from rbd, and attatch one rbd volume with 100G size to it. The
qemu and the osd is on the same host.
The fio is
[global]
ioengine=libaio
filename=/dev/vdb
thread
group_reporting
direct=1
iodepth=32
[write_32k]
bs=32k
rw=write
runtime=30
stonewall
[rand_write_4k]
bs=4k
rw=randwrite
runtime=30
stonewall
the result is
# origin
address@hidden test]# fio test.fio
write_32k: (g=0): rw=write, bs=32K-32K/32K-32K/32K-32K,
ioengine=libaio, iodepth=32
rand_write_4k: (g=1): rw=randwrite, bs=4K-4K/4K-4K/4K-4K,
ioengine=libaio, iodepth=32
fio-2.2.8
Starting 2 threads
Jobs: 1 (f=1): [_(1),w(1)] [67.0% done] [0KB/1342KB/0KB /s] [0/335/0
iops] [eta 00m:30s]
write_32k: (groupid=0, jobs=1): err= 0: pid=1842: Wed Feb 22 16:13:51 2017
write: io=746496KB, bw=24851KB/s, iops=776, runt= 30039msec
slat (usec): min=2, max=853, avg= 8.45, stdev=10.68
clat (msec): min=21, max=557, avg=41.19, stdev=43.43
lat (msec): min=21, max=557, avg=41.20, stdev=43.43
clat percentiles (msec):
| 1.00th=[ 23], 5.00th=[ 23], 10.00th=[ 23], 20.00th=[ 23],
| 30.00th=[ 24], 40.00th=[ 24], 50.00th=[ 24], 60.00th=[ 24],
| 70.00th=[ 25], 80.00th=[ 58], 90.00th=[ 91], 95.00th=[ 108],
| 99.00th=[ 247], 99.50th=[ 306], 99.90th=[ 465], 99.95th=[ 553],
| 99.99th=[ 562]
bw (KB /s): min= 4622, max=43722, per=100.00%, avg=25227.04, stdev=9684.02
lat (msec) : 50=79.46%, 100=14.34%, 250=5.32%, 500=0.82%, 750=0.06%
cpu : usr=0.48%, sys=0.65%, ctx=2278, majf=0, minf=6
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=23328/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=32
rand_write_4k: (groupid=1, jobs=1): err= 0: pid=1847: Wed Feb 22 16:13:51 2017
write: io=45668KB, bw=1519.8KB/s, iops=379, runt= 30063msec
slat (usec): min=1, max=682, avg= 8.05, stdev=12.10
clat (msec): min=8, max=549, avg=84.24, stdev=54.92
lat (msec): min=8, max=549, avg=84.25, stdev=54.92
clat percentiles (msec):
| 1.00th=[ 17], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 39],
| 30.00th=[ 72], 40.00th=[ 79], 50.00th=[ 86], 60.00th=[ 93],
| 70.00th=[ 95], 80.00th=[ 102], 90.00th=[ 112], 95.00th=[ 161],
| 99.00th=[ 326], 99.50th=[ 367], 99.90th=[ 486], 99.95th=[ 486],
| 99.99th=[ 553]
bw (KB /s): min= 396, max= 4548, per=100.00%, avg=1530.29, stdev=860.64
lat (msec) : 10=0.09%, 20=17.75%, 50=2.93%, 100=54.92%, 250=21.83%
lat (msec) : 500=2.48%, 750=0.02%
cpu : usr=0.23%, sys=0.45%, ctx=5636, majf=0, minf=1
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=11417/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: io=746496KB, aggrb=24850KB/s, minb=24850KB/s, maxb=24850KB/s,
mint=30039msec, maxt=30039msec
Run status group 1 (all jobs):
WRITE: io=45668KB, aggrb=1519KB/s, minb=1519KB/s, maxb=1519KB/s,
mint=30063msec, maxt=30063msec
Disk stats (read/write):
vdb: ios=122/34652, merge=0/0, ticks=69/1911324, in_queue=1913516, util=99.54%
#the scatter io is
address@hidden test]# fio test.fio
write_32k: (g=0): rw=write, bs=32K-32K/32K-32K/32K-32K,
ioengine=libaio, iodepth=32
rand_write_4k: (g=1): rw=randwrite, bs=4K-4K/4K-4K/4K-4K,
ioengine=libaio, iodepth=32
fio-2.2.8
Starting 2 threads
Jobs: 1 (f=1): [_(1),w(1)] [67.0% done] [0KB/1288KB/0KB /s] [0/322/0
iops] [eta 00m:30s]
write_32k: (groupid=0, jobs=1): err= 0: pid=1841: Wed Feb 22 16:21:51 2017
write: io=746496KB, bw=24837KB/s, iops=776, runt= 30056msec
slat (usec): min=2, max=183, avg= 8.48, stdev= 9.17
clat (msec): min=21, max=474, avg=41.22, stdev=42.54
lat (msec): min=21, max=474, avg=41.23, stdev=42.54
clat percentiles (msec):
| 1.00th=[ 23], 5.00th=[ 23], 10.00th=[ 23], 20.00th=[ 24],
| 30.00th=[ 24], 40.00th=[ 24], 50.00th=[ 24], 60.00th=[ 24],
| 70.00th=[ 25], 80.00th=[ 58], 90.00th=[ 95], 95.00th=[ 108],
| 99.00th=[ 225], 99.50th=[ 314], 99.90th=[ 453], 99.95th=[ 474],
| 99.99th=[ 474]
bw (KB /s): min= 5466, max=43786, per=100.00%, avg=25117.63, stdev=9332.73
lat (msec) : 50=79.86%, 100=13.43%, 250=5.92%, 500=0.78%
cpu : usr=0.41%, sys=0.76%, ctx=2614, majf=0, minf=6
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=23328/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=32
rand_write_4k: (groupid=1, jobs=1): err= 0: pid=1846: Wed Feb 22 16:21:51 2017
write: io=44980KB, bw=1496.7KB/s, iops=374, runt= 30055msec
slat (usec): min=1, max=489, avg= 8.07, stdev= 9.00
clat (msec): min=10, max=514, avg=85.51, stdev=57.00
lat (msec): min=10, max=514, avg=85.52, stdev=57.00
clat percentiles (msec):
| 1.00th=[ 17], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 43],
| 30.00th=[ 71], 40.00th=[ 83], 50.00th=[ 86], 60.00th=[ 93],
| 70.00th=[ 95], 80.00th=[ 102], 90.00th=[ 119], 95.00th=[ 176],
| 99.00th=[ 322], 99.50th=[ 359], 99.90th=[ 510], 99.95th=[ 515],
| 99.99th=[ 515]
bw (KB /s): min= 389, max= 4792, per=100.00%, avg=1515.82, stdev=850.12
lat (msec) : 20=18.51%, 50=2.05%, 100=54.64%, 250=21.91%, 500=2.77%
lat (msec) : 750=0.12%
cpu : usr=0.27%, sys=0.44%, ctx=5926, majf=0, minf=1
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=11245/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: io=746496KB, aggrb=24836KB/s, minb=24836KB/s, maxb=24836KB/s,
mint=30056msec, maxt=30056msec
Run status group 1 (all jobs):
WRITE: io=44980KB, aggrb=1496KB/s, minb=1496KB/s, maxb=1496KB/s,
mint=30055msec, maxt=30055msec
Disk stats (read/write):
vdb: ios=122/34509, merge=0/0, ticks=74/1911712, in_queue=1913542, util=99.58%
I also use systemstap thread_indent to trace the function time consumed.
The no-scatter.txt file is the original, and the scatter.txt is the scatter-io.
2017-02-21 19:30 GMT+08:00 Stefan Hajnoczi <address@hidden>:
> On Tue, Feb 21, 2017 at 03:43:59PM +0800, Jaze Lee wrote:
>> If I test , which cache mode i should choose ?
>>
>> writeback? or none? or default?
>
> Both cache=none and cache=writeback should be tested since block/rbd.c
> changes behavior depending on BDRV_O_NOCACHE.
--
谦谦君子
no-scatter.txt
Description: Text document
scatter.txt
Description: Text document
- [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd, jazeltq, 2017/02/20
- [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd, jazeltq, 2017/02/21
- Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd, Jaze Lee, 2017/02/21
- Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd, Stefan Hajnoczi, 2017/02/21
- Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd,
Jaze Lee <=
- Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd, Kevin Wolf, 2017/02/22
- Message not available
- Message not available
- Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd, Jaze Lee, 2017/02/22
Re: [Qemu-block] [RFC v6] RBD: Add support readv,writev for rbd, Jeff Cody, 2017/02/23