Re: [PULL 15/42] tests/functional: enable pre-emptive caching of assets

[Daniel P . Berrangé]

On Thu, Nov 28, 2024 at 09:25:03AM -0800, Pierrick Bouvier wrote:
> On 11/28/24 01:34, Daniel P. Berrangé wrote:
> > On Wed, Nov 27, 2024 at 10:31:13AM -0800, Pierrick Bouvier wrote:
> > > On 11/27/24 01:06, Daniel P. Berrangé wrote:
> > > > On Tue, Nov 26, 2024 at 04:54:18PM -0600, Richard Henderson wrote:
> > > > > On 11/26/24 11:52, Thomas Huth wrote:
> > > > > > I think we want to continue to maek failing downloads as test 
> > > > > > failures,
> > > > > > otherwise we'll never notice when an asset is not available from the
> > > > > > internet anymore (since SKIPs just get ignored).
> > > > > 
> > > > > I disagree.  Download failures are not rare.
> > > > 
> > > > Failures of the test to download assets will be rare *if* we have the
> > > > CI runner cache fixed. We only need to successfully download each
> > > > asset once, and it should be cached forever with no expiry timeout.
> > > > 
> > > > So we have an initially bootstrapping problem once caching is fixed,
> > > > where download failures could impact us. Once the cache is primed,
> > > > we'll only be at risk of download failures when introducing new
> > > > asset URLs, so I think it is fair to say failures should be rare
> > > > *if* we get the caching fixed.
> > > > 
> > > > With regards,
> > > > Daniel
> > > 
> > > Beyond the QEMU CI, we should think about users trying to run tests, and
> > > having the same kind of problems, but without having access to the magic
> > > cache.
> > > 
> > > Regarding the assets download, why don't we mirror them somewhere reliable
> > > instead of relying on third party storage?
> > 
> > If QEMU hosts these files, then QEMU is liable for license compliance,
> > IOW, we have to identify & potentially host the full & corresponding
> > source for all binaries in the images. This is not a business we want
> > to be involved in as a project.
> > 
> 
> That's interesting to know.
> In this case, pointing the original link origin with the artifact is not
> enough?

I wouldn't assume the origin is actually fully complying with the license
requirements to provide the source for all the assets we're consuming. By
not hosting binaries ourselves, we avoid any need to solve this problem
ourselves.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|