[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] make check speed
From: |
Lukáš Doktor |
Subject: |
Re: [Qemu-devel] make check speed |
Date: |
Wed, 23 Aug 2017 13:51:33 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 |
Dne 23.8.2017 v 10:35 Thomas Huth napsal(a):
> On 23.08.2017 10:01, Paolo Bonzini wrote:
>> On 23/08/2017 09:49, Thomas Huth wrote:
>>> While we're at it: I'd like to have a "make check-fast", too. Sometimes
>>> the normal "make check" is already too slow, e.g. while developing new
>>> patches, I sometimes just want to do a very quick sanity test to see
>>> whether I broke some basic things or not, and only do the "make check"
>>> before I submit my patches.
>>> So we would get three stages:
>>>
>>> - make check-fast => For very, very quick sanity tests only
>>>
>>> - make check => E.g. has to be run before submitting patches
>>>
>>> - make check-harder => might run a very long time, so best suited for
>>> nightly regression tests etc.?
>>>
>>> Does that sound reasonable? And the crucial question: Who is going to
>>> implement the basic framework for this?
>>
>> There's already make check-unit or make check-qtest-x86_64 depending on
>> what you're working on.
>
> True. And I just learned that you can also already set the SPEED
> variable to either "quick" or "slow" and that we're already using
> g_test_quick() and g_test_slow() in a couple of places to check this. So
> the framework for running quick vs. thorough tests is already there ...
> we just might want to add this to some more tests, I guess...
>
> Question for the maintainers and the test automation folks: Is anybody
> already running "make check SPEED=slow" or is this just rather an
> unheard-of way of running the tests?
>
>> If you have a many-core machine, of course, there's no simpler solution
>> than throwing more CPUs at it. :)
>
> Is it safe nowadays to run "make check -j4" for example? Last time I
> tried (maybe 1 or 2 years ago), there were still issues since some tests
> were using hard-coded temporary file names, so the parallel tests were
> disturbing each other, for example...
>
Actually the `.travis.yml` defines `MAKEFLAGS="-j3"`, which results in `make
check` being executed with 3 threads...
I was actually looking at the increasing number of failed travis builds and it
seems to be related to the fluctuating performance. Running `nice -n 20 make
check -j 12` with `nice -n 5 stress -c 20` in background results in the same
kind of failures:
File '/tmp/qemu/include/qapi/qmp/qobject.h'
Lines executed:0.00% of 9
**
ERROR:tests/prom-env-test.c:42:check_guest_memory: assertion failed
(signature == MAGIC): (0x7c7f1b78 == 0xcafec0de)
GTester: last random seed: R02S117a2b838f1fd17c65a134629577b996
make: *** [check-qtest-ppc] Chyba 1
/tmp/qemu/tests/Makefile.include:836: návod pro cíl „check-qtest-ppc“ selhal
make: *** [check-qtest-sparc] Chyba 1
/tmp/qemu/tests/Makefile.include:836: návod pro cíl „check-qtest-sparc“
selhal
**
ERROR:tests/rtc-test.c:173:check_time: assertion failed (ABS(t - s) <=
wiggle): (3 <= 2)
GTester: last random seed: R02S2ca82eb2e7f63ec62c8433b715d9fe12
Vhost user backend fails to broadcast fake RARP
**
ERROR:tests/vhost-user-test.c:779:test_reconnect: child process
(/i386/vhost-user/reconnect/subprocess [5264]) failed unexpectedly
GTester: last random seed: R02S6e728ccb5384a4d856cacc4166be8052
Gcov report for hw/misc/tmp105.c:
File 'hw/misc/tmp105.c'
Lines executed:30.40% of 125
Gcov report for arm-softmmu/hw/block/virtio-blk.c:
File '/tmp/qemu/hw/block/virtio-blk.c'
and all ERROR lines:
ERROR:tests/prom-env-test.c:42:check_guest_memory: assertion failed
(signature == MAGIC): (0x00000000 == 0xcafec0de)
ERROR:tests/rtc-test.c:173:check_time: assertion failed (ABS(t - s) <=
wiggle): (3 <= 2)
ERROR:tests/vhost-user-test.c:779:test_reconnect: child process
(/i386/vhost-user/reconnect/subprocess [5264]) failed unexpectedly
ERROR:tests/vhost-user-test.c:807:test_connect_fail: child process
(/x86_64/vhost-user/connect-fail/subprocess [7751]) failed unexpectedly
ERROR:tests/vhost-user-test.c:835:test_flags_mismatch: child process
(/x86_64/vhost-user/flags-mismatch/subprocess [7892]) failed unexpectedly
ERROR:tests/boot-sector.c:127:boot_sector_test: assertion failed (signature
== SIGNATURE): (0x00000000 == 0x0000dead)
on my machine. Using `make -j 1` or at least `make -j 2` could improve (but I
haven't checked that)
Lukáš
> Thomas
>
signature.asc
Description: OpenPGP digital signature
- Re: [Qemu-devel] [PATCH for-2.10] boot-serial-test: prefer tcg accelerator, (continued)
- Re: [Qemu-devel] [PATCH for-2.10] boot-serial-test: prefer tcg accelerator, Cornelia Huck, 2017/08/22
- Re: [Qemu-devel] [PATCH for-2.10] boot-serial-test: prefer tcg accelerator, David Gibson, 2017/08/22
- Re: [Qemu-devel] [PATCH for-2.10] boot-serial-test: prefer tcg accelerator, Cornelia Huck, 2017/08/23
- [Qemu-devel] make check speed (was: Re: [PATCH for-2.10] boot-serial-test: prefer tcg accelerator), Thomas Huth, 2017/08/23
- Re: [Qemu-devel] make check speed, Paolo Bonzini, 2017/08/23
- Re: [Qemu-devel] make check speed, Thomas Huth, 2017/08/23
- Re: [Qemu-devel] make check speed, Fam Zheng, 2017/08/23
- Re: [Qemu-devel] make check speed, Fam Zheng, 2017/08/23
- Re: [Qemu-devel] make check speed, Paolo Bonzini, 2017/08/23
- Re: [Qemu-devel] make check speed, David Gibson, 2017/08/23
- Re: [Qemu-devel] make check speed,
Lukáš Doktor <=
- Re: [Qemu-devel] make check speed, Thomas Huth, 2017/08/23
- Re: [Qemu-devel] make check speed, Lukáš Doktor, 2017/08/23
- Re: [Qemu-devel] make check speed, Cleber Rosa, 2017/08/24
- Re: [Qemu-devel] make check speed, Thomas Huth, 2017/08/23
- Re: [Qemu-devel] make check speed, Lukáš Doktor, 2017/08/23
- Re: [Qemu-devel] make check speed, Cornelia Huck, 2017/08/23
- Re: [Qemu-devel] make check speed, Thomas Huth, 2017/08/24
- Re: [Qemu-devel] make check speed, Cornelia Huck, 2017/08/30
- Re: [Qemu-devel] make check speed, Thomas Huth, 2017/08/30
- Re: [Qemu-devel] make check speed, Markus Armbruster, 2017/08/30