Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

Christopher Baines <address@hidden> writes:

> I've now written a very rough package and service for Patchwork [4], and
> managed to setup a instance here [5]. With the help of an email account
> subscribed to both guix-patches and guix-commits, getmail and a couple
> of scripts, it should also collect new patches sent to guix-patches, and
> mark those that have been merged to the master branch as "Accepted" [6].
>
> 4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33185
> 5: https://patchwork.cbaines.net/
> 6: https://patchwork.cbaines.net/project/guix-patches/list/?state=3

So, this has now moved forward a bit since this update. There is a
Summary at the bottom which you might want to skip ahead to.

I'm now testing out 3 services working together for testing patches:
 - https://patchwork.cbaines.net/
 - https://patchwork-fdo.cbaines.net/
 - https://laminar.cbaines.net/

Previously, I was just running Patchwork, and it was just picking up
patches from the mailing list, and marking committed patches as
accepted.

I mailed the Patchwork mailing list [1], as I had a question about
re-sending patches, and one thing to come out of that was the
realisation that I'd managed to stumble upon the documentation for a
fork of Patchwork.

1: https://lists.ozlabs.org/pipermail/patchwork/2018-November/005674.html

It turns out that there is a FreeDesktop fork [2][3][4]. It's
significant, as I think it does some things better than the upstream
version of Patchwork. One example is the Series page. For easy
comparison, I'm now running both, so [5] is upstream patchwork, and [6]
is the FreeDesktop fork.

2: https://patchwork.freedesktop.org/
3: https://gitlab.freedesktop.org/patchwork-fdo/patchwork-fdo/
4: https://patchwork-freedesktop.readthedocs.io/en/latest/

5: https://patchwork.cbaines.net/
6: https://patchwork-fdo.cbaines.net/

In addition to that, I wanted to continue working towards testing the
patches. Now Patchwork doesn't provide a framework for that, it just
provides a way to record the results of the testing. Maybe Cuirass can
do this in the future, but for now, I went with writing some bash
scripts, using some software called Laminar [1] as a useful
framework. It's tagline is "Continuous Integration the Light Way", and
it's something I've been wanted to try out for a little while.

7: https://laminar.ohwg.net/

I've configured 3 jobs [8] per patchwork service (fdo is short for
FreeDesktop):

 - patchwork(-fdo)-test-series:
   To test an individual series of patches.

 - patchwork(-fdo)-test-new:
   To trigger the testing of a series for all new (untested or updated)
   series.

 - patchwork(-fdo)-test-all:
   To trigger the testing of all open series, useful when the master
   branch of guix is updated.

8: https://laminar.cbaines.net/jobs

Laminar is configured by some files in /var/lib/laminar/cfg, so I've
added some NGinx configuration to make that public here [9]. Currently,
the patchwork(-fdo)-test-series jobs just:
 - Updates it's copy of the guix repository
 - Downloads the mbox from the relevant patchwork service
 - Tries applying it
 - Reports what happened to the relevant patchwork service

9: https://laminar.cbaines.net/cfg/

=== Summary ===

This is still very much a prototype, but I think it's nearing the point
where it actually might be doing something useful.

Currently, taking the service running the FreeDesktop fork of Patchwork,
you can go to the series page [10], and it will show you a list of patch
series, along with a indication of the "tests" status, which currently
is just whether the patch applied successfully to the master branch.

10: https://patchwork-fdo.cbaines.net/project/guix-patches/series/

The Patchwork service running the upstream code is very similar, you
just have to click in to the individual patches to get the information
on the "Checks", for an example, have a look at [11]

11: https://patchwork.cbaines.net/patch/233/

Next, I'm planning to work on extending the test series job to actually
start running some code, maybe perhaps just applying the patch, and then
running make at first. Then maybe trying to use the (guix inferior)
module code to intelligently determine changed packages that need
building. If you have something that does this, or want to write
something to do this, that would be awesome.

Any questions or comments, just let me know,

Chris

signature.asc
Description: PGP signature