Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Amnon Ilan]

----- Original Message -----
> From: "Wei Xu" <address@hidden>
> To: "Aaron Conole" <address@hidden>, "Flavio Leitner" <address@hidden>
> Cc: "Michal Privoznik" <address@hidden>, address@hidden, address@hidden, 
> address@hidden, "amit
> shah" <address@hidden>, "Amnon Ilan" <address@hidden>
> Sent: Tuesday, June 14, 2016 11:03:43 AM
> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open 
> from outside for unix socket
> 
> On 2016年06月09日 05:48, Aaron Conole wrote:
> > Flavio Leitner <address@hidden> writes:
> >
> >> Adding Aaron who is fixing exactly that on the OVS side.
> >>
> >> Aaron, please see the last question in the bottom of this email.
> >>
> >> On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Michal Privoznik" <address@hidden>
> >>>> To: "Daniel P. Berrange" <address@hidden>
> >>>> Cc: address@hidden, "amit shah" <address@hidden>,
> >>>> address@hidden, "Wei Xu" <address@hidden>,
> >>>> address@hidden
> >>>> Sent: Thursday, June 2, 2016 2:38:53 PM
> >>>> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
> >>>> 'fd' open from outside for unix socket
> >>>>
> >>>> On 02.06.2016 10:29, Daniel P. Berrange wrote:
> >>>>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> >>>>>> On 01.06.2016 18:16, Wei Xu wrote:
> >>>>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> >>>>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, address@hidden wrote:
> >>>>>>>>> From: Wei Xu <address@hidden>
> >>>>>>>>>
> >>>>>>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
> >>>>>>>>> create a unix socket for a chardev when managing VMs with libvirt,
> >>>>>>>>> because qemu don't have sufficient permissions in this case, and
> >>>>>>>>> proposal from libvirt team is opening the 'fd' in libvirt and
> >>>>>>>>> merely
> >>>>>>>>> passing it to qemu.
> >>>>>>>>
> >>>>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
> >>>>>>>> configuration
> >>>>>>>> of the guest. It is entirely possible for QEMU to create a unix
> >>>>>>>> socket
> >>>>>>>> - not
> >>>>>>>> least because that is exactly what QEMU uses for its QMP monitor
> >>>>>>>> backend.
> >>>>>>>> Looking at your example command line, I think the issue is simply
> >>>>>>>> that
> >>>>>>>> you
> >>>>>>>> should be putting the sockets in a different location. ie at
> >>>>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> >>>>>>>> permission to
> >>>>>>>> create sockets already.
> >>>>>>> ah.. adjusting permission or file location can solve this problem,
> >>>>>>> i'm
> >>>>>>> guessing maybe this is a more security concern, the socket is used as
> >>>>>>> a
> >>>>>>> network interface for a vm, similar as the qcow image file, thus
> >>>>>>> should
> >>>>>>> prevent it to be arbitrarily accessed.
> >>>>>>>
> >>>>>>> Michael, do you have any comment on this?
> >>>>>>
> >>>>>> I haven't seen the patches. But in libvirt we allow users to create a
> >>>>>> vhostuser interface and even specify where the socket should be
> >>>>>> placed:
> >>>>>>
> >>>>>>      <interface type='vhostuser'>
> >>>>>>        <mac address='52:54:00:ee:96:6c'/>
> >>>>>>        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
> >>>>>>        <model type='virtio'/>
> >>>>>>      </interface>
> >>>>>>
> >>>>>> The following cmd line is generated by libvirt then:
> >>>>>>
> >>>>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> >>>>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> >>>>>> -device
> >>>>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> >>>>>>
> >>>>>> Now, if we accept only /var/run/openvwitch path in
> >>>>>> /interface/source/@path (or whatever path to OVS is), we don't need
> >>>>>> this
> >>>>>> and have users manually label the dir (unless already labeled). But
> >>>>>> since we accept just any path in there, we should make sure that qemu
> >>>>>> is
> >>>>>> then able to create the socket. One possible fix would be to allow
> >>>>>> qemu
> >>>>>> create sockets just anywhere in the system. This, however, brings huge
> >>>>>> security risks and it's not acceptable IMO. The other option would be
> >>>>>> that libvirt would create the socket, and pass its FD to qemu (since
> >>>>>> libvirt already is allowed to create sockets anywhere).
> >>>>>
> >>>>> There are plenty of other places where we allow arbitrary paths in the
> >>>>> XML, but which have restrictions imposed by the security drivers. Not
> >>>>> least the <channel> devices which have the exact same scenario as this
> >>>>> network device, and require use of /var/lib/libvirt/qemu as the
> >>>>> directory
> >>>>> for the sockets. We certainly do not want to allow QEMU to create
> >>>>> sockets
> >>>>> anywhere.
> >>>>>
> >>>>> I don't think we want to grant QEMU svirtt permission to create sockets
> >>>>> in the /var/run/openvswitch directory either really.IMHO, users of
> >>>>> vhost
> >>>>> user should really be using /var/lib/libvirt/qemu, as is used for all
> >>>>> other UNIX sockets we create wrt other devices.
> >>>>
> >>>> Okay. I can live with that; but in that case we should document it
> >>>> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
> >>>> accessible and for the rest we do our best but maybe require sys admin
> >>>> intervention (e.g. to label the whole tree for a non-standard location).
> >>>
> >>> Does OVS has some limit for it's sockets to be only in
> >>> /var/run/openvswitch ?
> >
> > As of a recent commit, it can only be in /var/run/openvswitch or a
> > subdirectory therein (found in the openvswitch database).
> Aaron, thanks for your reply.
> 
> Just a question about the usage of openvswitch, in this user case when
> launching a vhostuser/dpdk via libvirt, qemu works as server mode for
> socket under /var/run/openvswitch, but per my previous test, ovs/dpdk
> always works as server mode, which means ovs will creates the socket and
> listening for connection, thus qemu works as client mode, does ovs/dpdk
> support working in client mode? which means it's qemu's duty to create
> the socket? and ovs will connect to it on demanding?

AFAIK qemu supports both client mode and server mode (configurable), so 
the solution should support both cases.

Thanks,
Amnon

> 
> >
> >>> Flavio, do you know?
> >>> If not, we are good as it is today with /var/lib/libvirt/qemu, right?
> >
> > Probably not.  There are other issues as well.  From a DAC perspective
> > (so forgetting selinux at the moment), qemu and ovs run as different
> > users.  This will mean that when ovs creates the unix domain socket
> > file, qemu will not be allowed to actually open it properly.  I have a
> > commit I'm trying to get upstream for that particular issue (see
> > bz:1281911 and upstream list discussion:
> >    http://openvswitch.org/pipermail/dev/2016-May/071453.html
> >    and
> >    http://openvswitch.org/pipermail/dev/2016-June/071978.html)
> >
> > Ansis is (I think) attacking this from the selinux/MAC side.  It would
> > be great to hear from users, libvirt folks, or anyone else in that
> > thread to help push it to a conclusion one way or another (even if the
> > approach that I've proposed is crap and wrong - then say it there :).
> >
> > Hope this helps,
> > -Aaron
> >
> 
>