Re: [Qemu-devel] Status and RFC of patchew testings on QEMU

[Daniel P. Berrange]

On Mon, Jul 17, 2017 at 02:35:21PM +0800, Fam Zheng wrote:
> Hi all,
> 
> Today I've included a fourth type of the automatic patchew replies: FreeBSD.
> 
> So far we have these tests running by patchew on each patch series:
> 
>   * Docker tests
>     Basically it is
>         make address@hidden \
>              address@hidden \
>              address@hidden"
> 
>   * checkpatch.pl
>     Each patch is fed to ./scripts/checkpatch.pl and all errors are reported.
> 
>   * s390x
>     It runs on a machine shared by Fedora team, basically only "./configure 
> and
>     make", because "make check" hanging is tricky to deal with from an
>     automation perspective. (Ideas?)
> 
>   * FreeBSD
>     Like s390x.
> 
> Q1: In the worst case, you get four individual auto replies from patchew. Is
> that too many? Do you prefer one reply with all the results concatenated into
> one?

The risk with combining everything into one reply is that if only one of the
four test systems fails / hangs / gets very backlogged, you're going to
delay reporting of failures from all four systems. Personally, I would like
to see any failure reported as soon as it happens, without waiting to see
if other things fail or pass.

> Q2: Some think the full log in the mail body is more than necessary. Is it
> better or worse if it is a "tail -n 200" of the log in the body and the full 
> log
> attached?

The mails are rather large alredy. It is common with Jenkins to only report
the tail of the logs, and then provide a hyperlink to a web site with the
complete log. This avoids bloating everyone's INBOXs with many 100 KB of
logs.

> Q3: What other tests do maintainers want? Different hosts? Different configure
> combinations?
> 
> Q4: Any other improvements/features you want? (E.g. some documentation? :)

Sometimes the test system seems to get pretty backlogged and there's no
way of knowing this, as its indistinguishable from the situation where
it doesn't send results because everything passed.

I'd like to see a web page that provides a list of all mail threads that
the test system has queued, with status of which jobs and running, and
once completed, provides the full logs.

That way we can quickly check whether patchw has started processing a
particular series or not.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|