Re: deferred cancellation of ipc

[Niels Möller]

Marcus Brinkmann <address@hidden> writes:

> On Tue, Oct 14, 2003 at 09:44:35AM +0200, Niels M?ller wrote:
> > Can one use the exchange registers to fake a function call in the
> > target thread? It should be possible to change the IP I guess, but
> > perhaps harder to save the return address.
> 
> Yes, that's one of the things ExchangeRegisters is good for.

Cool. How do you deal with the return address? [ It's fine with me if
you don't feel like digging into this detail now, I'm just curious
about how it works ]. Hmm, as long as the calling thread is in the
same address space, I guess one could (i) stop the target thread from
executing (how?), (ii) call ExchangeRegisters to mung the IP and
perhaps SP (iii) store the old IP somewhere in the address space, and
(iv) let the thread run again. But perhaps there's a simpler way to do
it?

> > If the thread is cancelled after the l4_ipc
> > system call has returned to user space, but before the line marked X
> > above, the returned value from the ipc is lost and the caller will
> > think it was cancelled.
> 
> Yes, that popped up in some of the things I considered as well.
> 
> > What we need here is for in_hurd_ipc to cleared atomically, as the
> > l4_ipc system call remains; almost anything that's tied to thread
> > ipc operations could work; a flag that's cleared on ipc
> > completion, or a counter of completed ipc, ...

I think it may be possible to distinguish between these two cases, but
to do it one gets one's hands dirty in the architecture-specific
details of the L4-spec. Assume the l4_ipc system call puts its return
value into machine register X when returning, and that after a
cancelled ipc, X will contain a value L4_CANCEL iff the L4 ipc operation
was cancelled.

Then we know that our ipc call has completed, without cancellation,
iff X != L4_CANCEL. This check can be done either by the
hurd_cancel_hook, or after setjmp has returned. Hmm, it's probably
safer to do it in hurd_cancel_hook, since C longjmp might restore X.
This method boils downs to something like

  /* All of these should be thread local. */
  volatile int cancelled = 0;
  volatile int in_hurd_ipc = 0;
  volatile jmpbuf cancelbuf;

  /* Reserve register for this compilation unit. */
  register l4_word X_register __asm("%x")__;  

  hurd_ipc()
  {
    if (setjmp(&cancelbuf) != 0)
      {
      cancel;
        in_hurd_ipc = 0;
        return E_CANCEL;
      }

    X_register = L4_CANCEL;

    /* Now the jmpbuf and X are valid. Tell the cancel hook about it. */
    in_hurd_ipc = 1;
  
    if (cancelled)
      goto cancel;
  
    ret = l4_ipc(...);
    in_hurd_ipc = 0;  /* X */
  
    return ret;
  }
  
  /* Always "called" by ExchangeRegisters munging if IP */
  hurd_cancel_hook()
  {
    cancelled = 1;
  
    if (in_hurd_ipc && X_register == L4_CANCEL)
      longjmp(cancelbuf, 17);
    else
      /* Return and do nothing. May need something
         more complicated than return, if we can't get
         our return address on stack. */
      return;
  }

One could use some other magic value than L4_CANCEL, it doesn't even
have to be a constant (this is important, if the X register is used
for the *input* to the l4_ipc system call as well as for the output),
the important thing is that after we have initialized the X register
to a suitable value, it is modified by and *only* by the l4_ipc call
returning to userspace. This implies that we must make sure that
nothing else in the chain, modifies X, including library glue for
l4_ipc *and* the glue pointed to by the kernel interface page.

When we get this low-level, it seems like it might be best to write
hurd_ipc and hurd_cancel_hook directly in assembler.

It would be cleaner with for example a counter in the thread control
block which is incremented by the kernel for each ipc call.

Regards,
/Niels