Re: [Qemu-devel] spec, RFC: TLS support for NBDµ

[Wouter Verhelst]

Hi Markus,

On Tue, Oct 21, 2014 at 10:17:17AM +0200, Markus Armbruster wrote:
> Wouter Verhelst <address@hidden> writes:
> > On Mon, Oct 20, 2014 at 01:51:43PM +0200, Markus Armbruster wrote:
[...]
> >> Furthermore, STARTTLS is vulnerable to active attacks: if you can get
> >> between the peers, you can make them fall back to unencrypted silently.
> >> How do you plan to guard against that?
> >
> > As I've said before in this discussion, encryption downgrade attacks are
> > not specific to STARTTLS; as soon as you have have an "encrypted" and an
> > "unencrypted" variant of a protocol, that becomes a problem. After all,
> > if an attacker can modify the communication so that STARTTLS is filtered
> > out of the communication, they can most likely also redirect all traffic
> > to a decrypting/encrypting proxy.
> >
> > The only way to fix that is through userspace; make "opportunistic" TLS
> > (i.e., use it if available, but move on if not) difficult to achieve.
> >
> >> See also https://www.agwa.name/blog/post/starttls_considered_harmful
> >
> > A random blog post by an author who is speaking about STARTTLS in
> > general terms is not a good technical argument for why STARTTLS is a bad
> > idea in *this* specific case.
> 
> Misunderstanding.  I didn't mean to claim "STARTTLS is bad".  If I
> wanted to say that, I would've said it directly.  I was merely asking
> how you plan to guard against downgrade attacks.  I gather your advice
> is to make the client (QEMU) insist on TLS, and check the server's
> certificate.  Correct?

My advice is to give both client and server the ability to have TLS
switched on or off, and possibly (but not necessarily so, and certainly
not by default) also the _ability_ to negotiate TLS if the other side
supports it, while not aborting if it doesn't.

> > If I was defining a new protocol from scratch, I might dump the whole
> > thing in TLS to begin with. But that's just not the case, so I have to
> > deal with what exists already.
> 
> Yes.  You can still start over on a separate port.  However:

No, I can't, at least not if I want to continue to use an assigned port
without breaking backwards compatibility. When 10809 was assigned to
NBD, IANA made it perfectly clear that I wouldn't get a new port number
for a "secure" variant.

> > In addition, with the current state of affairs, it is *not possible* to
> > swap to an NBD device if you need to pipe its data through a separate
> > socket than the one you're handing to the kernel. The result of that is
> > that you can't do TLS on a device you want to swap to. This means we
> > need to continue to support a protocol that can do TLS for some exports,
> > and plain (unencrypted) traffic for other exports, *in the same running
> > nbd-server instance*.
> 
> I don't understand enough about NBD to challenge this argument.  The sad
> consequence is more complexity and a larger attack surface.
> 
> Out of curiosity: is this "merely" an implementation restriction, or is
> it more fundamental?

Well, to some extent, every software issue is merely an implementation
restriction...

The problem is that in order to clear memory when your swap device is on
the other side of a network connection, you need to write to said swap
device, which causes TCP traffic, which means the kernel needs to read
TCP ACK packets (or in the case of NBD, the reply packet to the
NBD_CMD_WRITE request that you sent out) to ensure that the traffic has
in fact reached its destination. Unfortunately, you can't impose
ordering on incoming network traffic, so that means you may need to read
through a whole lot of youtube MPEG traffic or some such in order to
find your one TCP ACK that tells you your swapout has been processed
correctly and you can now clear the memory which you wrote out to swap.

It's fairly obvious how that could lead to a deadlock if you don't know
that this one data stream is not related to swapspace while this other
one here is.

That's why a PF_MEMALLOC kernel-space socket option was created; sockets
marked with that option are related to a swapdevice, so when the kernel
is low on memory, it will throw away all incoming network traffic
_except_ for packets destined for a socket marked with that option, in
the hope that this will allow us to clear up some memory.

Since it's a kernel-space only thing, that means you can't mark sockets
as such from userspace; so if the NBD traffic is wrapped in some other
traffic from which it needs to be unwrapped in userspace, then the
userspace component would basically be a proxy with two sockets -- one
towards the server, one towards the kernel. The socket towards the
kernel would have the PF_MEMALLOC socket option set, but the one towards
the server would not, and *that* is the one which actually has data
flowing in over the network. There's your deadlock again.

Since, with the current state of affairs, the NBD handshake is done in
user space with the actual data pushing stuff later being done in kernel
space, that means you do actually need to unwrap the TLS traffic in
userspace.

I can't think of many ways that would allow us to get around that issue:
- Figure out a way to move negotiated keys from user space to kernel
  space, and handle the TLS in kernel space rather than user space. I'm
  not even sure if you _can_ do TLS in kernel space (let alone whether
  it's a good idea).
- Move the handshake to kernel space from user space. I'm not a fan of
  that idea. It would also still require much of the TLS in kernel
  space.
- Patch the kernel so that userspace can mark a particular socket with
  PF_MEMALLOC. And mlock() the whole TLS stack into memory (including
  keys etc).

All three of those require "patch the kernel", so at any rate going
there would require "wait until the kernel supports it before we can do
TLS", which I think is not a good option.

> > I did add the NBD_REP_ERR_TLS_REQD message for a server which does not
> > export anything unencrypted, so that it can (after the initial few
> > exchanges) reply to anything except for STARTTLS with "lalala, I'm not
> > listening until you encrypt things". However, unless it's fine to ditch
> > support for swapping to an NBD device (not an option from where I'm
> > standing), dropping support for unencrypted communication is not an
> > option.
> 
> That's a good idea.  It doesn't make things less complex, though.

True.

-- 
It is easy to love a country that is famous for chocolate and beer

  -- Barack Obama, speaking in Brussels, Belgium, 2014-03-26