Re: 'make check-acceptance' eats lots of disk space and never cleans it up

[Peter Maydell]

On Tue, 3 Aug 2021 at 13:58, Cleber Rosa <crosa@redhat.com> wrote:
>
> On Tue, Aug 3, 2021 at 8:43 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > It looks like 'make check-acceptance' creates directories in
> > build/clang/tests/results which are huge and which it never
> > cleans up. For example one of my build directories (configured
> > just for arm targets) has over 350 'job-[timestamp]' directories,
> > many of which are 2.5GB or more in size.
> >
>
> Every "job-[timestamp]" directory is the result of an "avocado run"
> invocation, that is, one "make check-acceptance" command.
>
> > I assume most of this is artefacts (disk images etc) needed to
> > rerun the tests. That's useful to keep around so you can manually
> > run a test. However, we should be sharing this between runs, not
> > creating a fresh copy for every time check-acceptance is
> > run, surely ?
> >
>
> They contain results and files needed for debugging the results of
> tests, not artefacts needed to re-run them.  Everything that is
> shareable is in the "~/avocado/data/caches" directory.

This doesn't in practice seem to be the case. Picking a subdirectory
at random:

./build/clang/tests/results/job-2021-07-30T11.20-63bd0a6/test-results/tmp_dir4_a3m36o/091-tests_acceptance_machine_sparc64_sun4u.py_Sun4uMachine.test_sparc64_sun4u/day23

This contains (among other things) a vmlinux file which I assume is
the one we run on the guest. It looks to me like this is a directory
where we unzipped/untarred a downloaded file with the guest image.

And another:

./build/clang/tests/results/job-2021-07-30T11.20-63bd0a6/test-results/tmp_dirwowk1bzp/026-tests_acceptance_boot_linux_console.py_BootLinuxConsole.test_arm_cubieboard_initrd/

This seems to contain a rootfilesystem for some test or other,
with a boot/, lib/, usr/, etc.

These all look like artefacts to me, in the sense that they're
the same every time.

I notice that all these have 'tmp_dir*' directories in the paths. Is the
problem just that we're failing to clean up a tempdir in some situations?

thanks
-- PMM