Re: [Qemu-devel] [PULL 4/4] tests/docker/Makefile.include: add a generic docker-run target

[Fam Zheng]

On Sat, 10/15 15:48, Alex Bennée wrote:
> 
> Paolo Bonzini <address@hidden> writes:
> 
> > On 14/10/2016 17:29, Fam Zheng wrote:
> >> From: Alex Bennée <address@hidden>
> >>
> >> This re-factors the docker makefile to include a docker-run target which
> >> can be controlled entirely from environment variables specified on the
> >> make command line. This allows us to run against any given docker image
> >> we may have in our repository, for example:
> >>
> >>     make docker-run TEST="test-quick" IMAGE="debian:arm64" \
> >>          EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> >>
> >> The existing address@hidden targets still work but the inline
> >> verification has been dropped because we already don't hit that due to
> >> other pattern rules in rules.mak.
> >>
> >> Signed-off-by: Alex Bennée <address@hidden>
> >>
> >> Message-Id: <address@hidden>
> >> Message-Id: <address@hidden>
> >> [Squash in the verification removal patch. - Fam]
> >> Signed-off-by: Fam Zheng <address@hidden>
> >> ---
> >>  tests/docker/Makefile.include | 61 
> >> +++++++++++++++++++++++++++----------------
> >>  1 file changed, 38 insertions(+), 23 deletions(-)
> >>
> >> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> >> index b44daab..925f711 100644
> >> --- a/tests/docker/Makefile.include
> >> +++ b/tests/docker/Makefile.include
> >> @@ -78,6 +78,7 @@ docker:
> >>    @echo '                         "IMAGE" is one of the listed container 
> >> name."'
> >>    @echo '    docker-image:        Build all images.'
> >>    @echo '    docker-image-IMAGE:  Build image "IMAGE".'
> >> +  @echo '    docker-run:          For manually running a "TEST" with 
> >> "IMAGE"'
> >>    @echo
> >>    @echo 'Available container images:'
> >>    @echo '    $(DOCKER_IMAGES)'
> >> @@ -101,31 +102,45 @@ docker:
> >>    @echo '    NOCACHE=1            Ignore cache when build images.'
> >>    @echo '    EXECUTABLE=<path>    Include executable in image.'
> >>
> >> +# This rule if for directly running against an arbitrary docker target.
> >> +# It is called by the expanded docker targets (e.g. make
> >> +# address@hidden) which will do additional verification.
> >> +#
> >> +# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" 
> >> EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> >> +#
> >> +docker-run: docker-qemu-src
> >> +  @mkdir -p "$(DOCKER_CCACHE_DIR)"
> >> +  @if test -z "$(IMAGE)" || test -z "$(TEST)"; \
> >> +          then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
> >> +  fi
> >> +  $(if $(EXECUTABLE),                                             \
> >> +          $(call quiet-command,                                   \
> >> +                  $(SRC_PATH)/tests/docker/docker.py update       \
> >> +                  $(IMAGE) $(EXECUTABLE),                         \
> >> +                  "  COPYING $(EXECUTABLE) to $(IMAGE)"))
> >> +  $(call quiet-command,                                           \
> >> +          $(SRC_PATH)/tests/docker/docker.py run                  \
> >> +                  -t                                              \
> >> +                  $(if $V,,--rm)                                  \
> >> +                  $(if $(DEBUG),-i,--net=none)                    \
> >> +                  -e TARGET_LIST=$(TARGET_LIST)                   \
> >> +                  -e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
> >> +                  -e V=$V -e J=$J -e DEBUG=$(DEBUG)               \
> >> +                  -e SHOW_ENV=$(SHOW_ENV)                         \
> >> +                  -e CCACHE_DIR=/var/tmp/ccache                   \
> >> +                  -v $$(readlink -e 
> >> $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> >> +                  -v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z       \
> >> +                  $(IMAGE)                                        \
> >> +                  /var/tmp/qemu/run                               \
> >> +                  $(TEST), "  RUN $(TEST) in ${IMAGE}")
> >> +
> >> +# Run targets:
> >> +#
> >> +# Of the form address@hidden which will then be expanded into a call to 
> >> "make docker-run"
> >>  docker-run-%: CMD = $(shell echo '$@' | sed -e 
> >> 's/docker-run-\(address@hidden)@\(.*\)/\1/')
> >>  docker-run-%: IMAGE = $(shell echo '$@' | sed -e 
> >> 's/docker-run-\(address@hidden)@\(.*\)/\2/')
> >> -docker-run-%: docker-qemu-src
> >> -  @mkdir -p "$(DOCKER_CCACHE_DIR)"
> >> -  @if test -z "$(IMAGE)" || test -z "$(CMD)"; \
> >> -          then echo "Invalid target"; exit 1; \
> >> -  fi
> >> -  $(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
> >> -          $(call quiet-command,\
> >> -                  if $(SRC_PATH)/tests/docker/docker.py images | \
> >> -                          awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} 
> >> END{exit(!found)}'; then \
> >> -                          $(SRC_PATH)/tests/docker/docker.py run $(if 
> >> $V,,--rm) \
> >> -                          -t \
> >> -                          $(if $(DEBUG),-i,--net=none) \
> >> -                          -e TARGET_LIST=$(TARGET_LIST) \
> >> -                          -e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) 
> >> \
> >> -                          -e V=$V -e J=$J -e DEBUG=$(DEBUG) -e 
> >> SHOW_ENV=$(SHOW_ENV)\
> >> -                          -e CCACHE_DIR=/var/tmp/ccache \
> >> -                          -v $$(readlink -e 
> >> $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> >> -                          -v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
> >> -                          qemu:$(IMAGE) \
> >> -                          /var/tmp/qemu/run \
> >> -                          $(CMD); \
> >> -                  fi \
> >> -                  ,"RUN","$(CMD) in $(IMAGE)")))
> >> +docker-run-%:
> >> +  @make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
> >
> > This "make" should be $(MAKE), so that some parameters are handled
> > correctly (e.g. -n/--dry-run) and especially so that parallel make works
> > correctly.

Parallism should terminate here because docker-run is a leaf target. But you are
right about parameters.

> 
> Fam,
> 
> Do you want me to re-spin and re-send or are you going to fix up in your
> pull request?

I'll fix up in my tree and send pull request again.

Fam

> 
> >
> > Thanks,
> >
> > Paolo
> >
> >>  docker-clean:
> >>    $(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
> >>
> 
> 
> --
> Alex Bennée