Re: [Qemu-devel] [PATCH v2 0/3] Support building qemu-user powered docker test images

[Riku Voipio]

On 13 June 2016 at 15:30, Alex Bennée <address@hidden> wrote:
>
> Riku Voipio <address@hidden> writes:
>
>> On 13 June 2016 at 12:22, Alex Bennée <address@hidden> wrote:
>>>
>>> Fam Zheng <address@hidden> writes:
>>>
>>>> On Wed, 06/08 17:35, Alex Bennée wrote:
>>>>> Hi,
>>>>>
>>>>> This is a re-spin of the previous series built on top of
>>>>> fam/docker.next. I've made the changes suggested in the last review
>>>>> and split the first patch apart to separate (and fix) the build
>>>>> directory changes first.
>>>>>
>>>>> Now it no longer messes with the docker file you can actually
>>>>> cross-build tests. First ensure you build the debian-bootstrap image:
>>>>>
>>>>>     DEB_ARCH=armhf DEB_TYPE=testing \
>>>>>       ./tests/docker/docker.py build qemu:debian-bootstrap \
>>>>>       ./tests/docker/dockerfiles/debian-bootstrap.docker \
>>>>>       --include-executable=./arm-linux-user/qemu-arm
>>>>
>>>> This can take long depending on the network - I had to explicitly use a 
>>>> close
>>>> mirror in the pre script to test this, not sure why the mirror redirector
>>>> doesn't work. Eventually I get this error once the pre script succeeds and
>>>> docker build starts:
>>>>
>>>> Sending build context to Docker daemon 235.1 MB
>>>> Step 1 : FROM scratch
>>>>  --->
>>>> Step 2 : ADD . /
>>>>  ---> 807bfa810b0c
>>>> Removing intermediate container e57ded00b227
>>>> Step 3 : RUN sed -i 's/in_target mount/echo not for docker in_target 
>>>> mount/g' /debootstrap/functions
>>>>  ---> Running in fd80232b38fc
>>>> rpc error: code = 2 desc = "oci runtime error: exec format error"
>>>>
>>>> IIUC the sed is an armhf binary? Is something missing in the docker file? I
>>>> cannot find where the copied qemu-arm is used...
>>>
>>> Ahh I realise I missed out the implicit binfmt_misc needs to be set up.
>>> So on my Ubuntu system just having qemu-user installed means the host
>>> system binfmt_misc is set up for armhf binaries to run
>>> /usr/bin/qemu-arm. I haven't experimented with explicitly setting up
>>> binfmt_misc in the container because it was already working :-/
>>
>> The containment leaks here. If your host has registered binfmt_misc,
>> identical settings will be visible in container. You can't change the
>> settings inside docker, unless you start with --privileged. And
>> changing binfmt_misc settings inside docker will also change the
>> settings in the host. So one needs to tiptoe around setting it up...
>
> Hmm well that makes it interesting. FWIW I'd stick to setting up binfmt
> outside the container and defaulting to the "just works" setup depending
> on how $HOST sets up binfmt_misc. The original patch actually queried
> binfmt_misc to work out the exact path it needed to install qemu-arm in
> the $GUEST container. For now we just default to /usr/bin which works on
> Debian-types without problem.

For some value of without problem - you have qemu-user installed, so
you have /usr/bin/qemu-arm registered, while I have qemu-user-static
installed and thus /usr/bin/qemu-arm-static registered.

We have the ./scripts/qemu-binfmt-conf.sh we could run if no binfmt's
appear registered, or at least print an error message for the enduser
to run it.

>>
>>>>
>>>> Fam
>>>>
>>>>>
>>>>> And then run the test quick target:
>>>>>
>>>>>     make address@hidden J=9 V=1
>>>>>
>>>>> I'll leave it up to you how we cleanly integrate multi-arch builds
>>>>> into the Make system ;-)
>>>>>
>>>>> Alex Bennée (3):
>>>>>   tests/docker/docker.py: docker_dir outside build
>>>>>   tests/docker/docker.py: support --include-executable
>>>>>   add debian-bootstrap.docker target (and pre script)
>>>>>
>>>>>  tests/docker/docker.py                           | 68 
>>>>> +++++++++++++++++++++---
>>>>>  tests/docker/dockerfiles/debian-bootstrap.docker | 21 ++++++++
>>>>>  tests/docker/dockerfiles/debian-bootstrap.pre    |  5 ++
>>>>>  3 files changed, 86 insertions(+), 8 deletions(-)
>>>>>  create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker
>>>>>  create mode 100755 tests/docker/dockerfiles/debian-bootstrap.pre
>>>>>
>>>>> --
>>>>> 2.7.4
>>>>>
>>>
>>>
>>> --
>>> Alex Bennée
>
>
> --
> Alex Bennée