Re: [Qemu-devel] [PATCH v2 0/3] Support building qemu-user powered docker test images

[Alex Bennée]

Riku Voipio <address@hidden> writes:

> On 13 June 2016 at 15:30, Alex Bennée <address@hidden> wrote:
>>
>> Riku Voipio <address@hidden> writes:
>>
>>> On 13 June 2016 at 12:22, Alex Bennée <address@hidden> wrote:
>>>>
>>>> Fam Zheng <address@hidden> writes:
>>>>
>>>>> On Wed, 06/08 17:35, Alex Bennée wrote:
>>>>>> Hi,
>>>>>>
>>>>>> This is a re-spin of the previous series built on top of
>>>>>> fam/docker.next. I've made the changes suggested in the last review
>>>>>> and split the first patch apart to separate (and fix) the build
>>>>>> directory changes first.
>>>>>>
>>>>>> Now it no longer messes with the docker file you can actually
>>>>>> cross-build tests. First ensure you build the debian-bootstrap image:
>>>>>>
>>>>>>     DEB_ARCH=armhf DEB_TYPE=testing \
>>>>>>       ./tests/docker/docker.py build qemu:debian-bootstrap \
>>>>>>       ./tests/docker/dockerfiles/debian-bootstrap.docker \
>>>>>>       --include-executable=./arm-linux-user/qemu-arm
>>>>>
>>>>> This can take long depending on the network - I had to explicitly use a 
>>>>> close
>>>>> mirror in the pre script to test this, not sure why the mirror redirector
>>>>> doesn't work. Eventually I get this error once the pre script succeeds and
>>>>> docker build starts:
>>>>>
>>>>> Sending build context to Docker daemon 235.1 MB
>>>>> Step 1 : FROM scratch
>>>>>  --->
>>>>> Step 2 : ADD . /
>>>>>  ---> 807bfa810b0c
>>>>> Removing intermediate container e57ded00b227
>>>>> Step 3 : RUN sed -i 's/in_target mount/echo not for docker in_target 
>>>>> mount/g' /debootstrap/functions
>>>>>  ---> Running in fd80232b38fc
>>>>> rpc error: code = 2 desc = "oci runtime error: exec format error"
>>>>>
>>>>> IIUC the sed is an armhf binary? Is something missing in the docker file? 
>>>>> I
>>>>> cannot find where the copied qemu-arm is used...
>>>>
>>>> Ahh I realise I missed out the implicit binfmt_misc needs to be set up.
>>>> So on my Ubuntu system just having qemu-user installed means the host
>>>> system binfmt_misc is set up for armhf binaries to run
>>>> /usr/bin/qemu-arm. I haven't experimented with explicitly setting up
>>>> binfmt_misc in the container because it was already working :-/
>>>
>>> The containment leaks here. If your host has registered binfmt_misc,
>>> identical settings will be visible in container. You can't change the
>>> settings inside docker, unless you start with --privileged. And
>>> changing binfmt_misc settings inside docker will also change the
>>> settings in the host. So one needs to tiptoe around setting it up...
>>
>> Hmm well that makes it interesting. FWIW I'd stick to setting up binfmt
>> outside the container and defaulting to the "just works" setup depending
>> on how $HOST sets up binfmt_misc. The original patch actually queried
>> binfmt_misc to work out the exact path it needed to install qemu-arm in
>> the $GUEST container. For now we just default to /usr/bin which works on
>> Debian-types without problem.
>
> For some value of without problem - you have qemu-user installed, so
> you have /usr/bin/qemu-arm registered, while I have qemu-user-static
> installed and thus /usr/bin/qemu-arm-static registered.
>
> We have the ./scripts/qemu-binfmt-conf.sh we could run if no binfmt's
> appear registered, or at least print an error message for the enduser
> to run it.

I noticed that has been updated recently. I shall have a look and see if
I can make the pre-script return an error if the binfmt conditions
aren't met. Hopefully it doesn't require root permissions to check all
this out ;-)

>
>>>
>>>>>
>>>>> Fam
>>>>>
>>>>>>
>>>>>> And then run the test quick target:
>>>>>>
>>>>>>     make address@hidden J=9 V=1
>>>>>>
>>>>>> I'll leave it up to you how we cleanly integrate multi-arch builds
>>>>>> into the Make system ;-)
>>>>>>
>>>>>> Alex Bennée (3):
>>>>>>   tests/docker/docker.py: docker_dir outside build
>>>>>>   tests/docker/docker.py: support --include-executable
>>>>>>   add debian-bootstrap.docker target (and pre script)
>>>>>>
>>>>>>  tests/docker/docker.py                           | 68 
>>>>>> +++++++++++++++++++++---
>>>>>>  tests/docker/dockerfiles/debian-bootstrap.docker | 21 ++++++++
>>>>>>  tests/docker/dockerfiles/debian-bootstrap.pre    |  5 ++
>>>>>>  3 files changed, 86 insertions(+), 8 deletions(-)
>>>>>>  create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker
>>>>>>  create mode 100755 tests/docker/dockerfiles/debian-bootstrap.pre
>>>>>>
>>>>>> --
>>>>>> 2.7.4
>>>>>>
>>>>
>>>>
>>>> --
>>>> Alex Bennée
>>
>>
>> --
>> Alex Bennée


--
Alex Bennée