Re: [Qemu-devel] State of QEMU CI as we enter 4.0

[Alex Bennée]

Peter Maydell <address@hidden> writes:

> On Thu, 14 Mar 2019 at 15:57, Alex Bennée <address@hidden> wrote:
>> Testing in the Cloud
>> ====================
>>
>> After BuildBot went out-of-service we have been relying heavily on Travis
>> as our primary CI platform. This has been creaking somewhat under the
>> strain and while we have a large test matrix its coverage is fairly
>> Ubuntu/x86 centric. However in recent months we've expanded and we now
>> have:
>>
>>   - Shippable, cross compilers - catches a lot of 32/64 bit isms
>>   - Cirrus, FreeBSD and MacOS builds
>>   - GitLab, Alternative x86/Debian - iotests
>
> Are any of these capable of replacing my ad-hoc collection
> of build test systems for testing merges ? I would quite like
> to be able to do that, because it would make it easier for
> other people to take over the process of handling pull requests
> when I'm away.
>
> I think the main requirements for that would be:
>  * covers full range of hosts[*]
>  * can be asked to do a test build of a merge before
>    I push it to master
>  * reliably completes all builds within say 90 minutes
>    of being asked to start
>
> [+] I currently test:
>  - windows crossbuilds

We did have this with shippable but had to disable it when the upstream
repo went down. We could re-enable if we can rebuild it and cache our
docker images with Daniel's work.

>  - S390, AArch32, AArch64, PPC64 Linux
>    (SPARC currently disabled because of the migration-test flakiness)

We would need to get machines from somewhere. Setting up a headless
SynQuacer should be easy enough and we have qemu-test which is a
ThunderX beast. I guess the IBM guys would have to chime in if they
could find PPC/s390 boxen because I'm guessing spamming the GCC build
farm with our test runners would be a little unfair.

>  - OSX

Currently run on Travis and recently Cirrus

>  - FreeBSD, OpenBSD, NetBSD via the tests/vm setup

We build on FreeBSD on Cirrus - but any x86 box can run the test/vm
setup assuming your just kicking it off with a make vm-test type thing?

>  - various x86-64 configs: from-clean; debug; clang; TCI; no-tcg;
>    linux-static (including 'make check-tcg')

This is already covered in our rather large Travis matrix. The trick
will be making it all fast enough.

--
Alex Bennée