Re: [Qemu-devel] [PATCH] docker: Fix PATH for ccache

[Daniel P. Berrange]

On Mon, Oct 23, 2017 at 02:22:37PM +0800, Fam Zheng wrote:
> On Fri, 10/20 13:51, Daniel P. Berrange wrote:
> > On Wed, Oct 18, 2017 at 09:35:49AM +0100, Daniel P. Berrange wrote:
> > > On Wed, Oct 18, 2017 at 04:28:36PM +0800, Fam Zheng wrote:
> > > > On Wed, 10/18 01:08, address@hidden wrote:
> > > > > Your branch is up-to-date with 'origin/test'.
> > > > > Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for 
> > > > > path 'dtc'
> > > > > Cloning into 
> > > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'...
> > > > > fatal: unable to connect to git.qemu-project.org:
> > > > > git.qemu-project.org[0: 172.99.69.163]: errno=Connection timed out
> > > > > 
> > > > > fatal: clone of 'git://git.qemu-project.org/dtc.git' into submodule 
> > > > > path 
> > > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'
> > > > >  failed
> > > > > Failed to clone 'dtc'. Retry scheduled
> > > > > Cloning into 
> > > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'...
> > > > > fatal: unable to connect to git.qemu-project.org:
> > > > > git.qemu-project.org[0: 172.99.69.163]: errno=Connection timed out
> > > > > 
> > > > > fatal: clone of 'git://git.qemu-project.org/dtc.git' into submodule 
> > > > > path 
> > > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'
> > > > >  failed
> > > > > Failed to clone 'dtc' a second time, aborting
> > > > > failed to init submodule dtc
> > > > 
> > > > So now patchew is less robust on network hiccups: a number of the same 
> > > > "git
> > > > submodule update --init" calls have succeeded above, so this is 
> > > > incidental. Any
> > > > idea how to improve that? Maybe if there is a checkout from $QEMU_SRC, 
> > > > clone/cp
> > > > from there instead of fetch from submodule remote?
> > > 
> > > If patchew has unreliable connectivity to git.qemu.org then the best thing
> > > is probably have a background job that mirrors the git.qemu.org repos on
> > > the patchew server. Then have the test job modify .gitmodules to point to
> > > your local mirror. That way, network access is used during the background
> > > job only, and actual builds just get a local clone from the cache.
> > 
> > I thought of a simpler way to address this.
> > 
> > Currently patchew runs 4 docker jobs, and each one triggers creation of
> > the $(DOCKER_SRC_COPY) rule. This runs archive-source.sh 4 times, because
> > $(DOCKER_SRC_COPY) contains the current timestamp and so is different
> > each time we run it.
> > 
> > If you override DOCKER_SRC_COPY with the current git change hash, then
> > you would end up using the same archive for all 4 jobs, and so only do
> > the checkouts once
> >  
> >   make docker-foo   DOCKER_SRC_COPY=docker-src.`git rev-parse HEAD`
> 
> I'm fine with that, just not sure about if this handles uncommitted changes
> correctly? It certainly isn't a concern on patchew, but may be a problem for
> developers.

Developers running locally wouldn't need to set this variable - they can
just run 'make docker-foo' and it'll do the right thing wrt git stash

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|