Re: A comment about changing kernels

[Jonathan S. Shapiro]

On Mon, 2005-10-31 at 19:06 +0100, Espen Skoglund wrote:

> Just to set the record straight.  I believe that your comments here
> are absolutely off target because:
> 
>   a) His kernel supported small spaces.  There's no way to implement
>      small spaces correctly without reloading segment registers.  In
>      other words: your statement is completely wrong.
> 
>   b) It makes absoultely no sense be paranoid about restricting the
>      communication channel resulting from reloading segment registers
>      when doing an IPC between two partners (i.e., you already have
>      the communication channel).  Avoiding such a communication
>      channel is only an issue when some other type of context switch
>      occurs (and this is not on the critical IPC path).
> 
> I know that you stated the same claims in your "Vulnerabilities..."
> paper, and so these claims are now on record as being the "truth".

Espen:

It really was a bug, and it has nothing to do with the IPC path, and it
has nothing to do with small spaces, so my statement may or may not be
wrong. Let me describe the actual bug:

Obviously, segment registers must be restored (both in IPC and in
trap/interrupt handling). The problem was that the particular L4
implementation in question did not *save* the segment registers on entry
into the kernel. In consequence it *couldn't* restore them. What it
actually did was set them unconditionally to the kernel-expected values
without regard to what the correct selector value was. [Note: This was
purely an implementation bug, not an architectural flaw.]

One effect of this is that a process could do the following:

        mov $0 -> %ax
        mov %ax -> %es  ; for example

1:      mov %es -> %ax
        jnz 1b

        ; when you get here, an interrupt has been
        ; detected.

That is: the L4 implementation assumed incorrectly that no value could
be loaded into the segment register other than the one that the OS
provided, and that in consequence there was no need to save the segment
register selector value. Unfortunately, the Pentium specification says
that the NULL selector load always succeeds (i.e. without any reference
to the corresponding GDT entry). I pointed this out to Jochen somewhere
around 2000, and he agreed that it was a bug.

After this null selector load, the situation is that the process can
detect preemption by detecting when the selector has been restored by
the interrupt handler path within the microkernel. This code path also
had another side effect, which is desirable from a performance
standpoint but still incorrect:

Process A stops. Process B uses debugging mechanisms to set process A CS
register to NULL. Process B will resume successfully instead of
faulting, because the selector value is restored without regard to what
the value was supposed to be.

The correct behavior was to save the segment selectors, and this adds
about 20 cycles to the path. This means that the reported small space
performance figure of 135 cycles/IPC was actually low by about 15%
relative to a correct implementation. So the situation was that the
(incorrect) L4 path was compared to the (correct) EROS path to suggest
equal performance, but a corrected L4 path would have been measurably
slower.

Definitely not an earth-shattering issue, and definitely not something
that would have any noticeable impact on overall systemic performance
(which is dominated by cache effects), but when any of us makes claims
of the form "IPCs are fast, take two", or "every cycle counts, and the
sessionless protocol is noticeably slower", we need to make sure that we
are measuring the costs correctly -- including all of the user-level
costs that are incurred by one implementation vs. another.

Please note: this comment definitely applies to my work as well, and it
is very hard to do.


shap