Re: Contribution to the Hurd on L4

[Marcus Brinkmann]

At Thu, 06 Jan 2005 21:22:15 +0100,
Matthieu Lemerre <address@hidden> wrote:
> 
> Marcus Brinkmann <address@hidden> writes:
> 
> > At Thu, 30 Dec 2004 19:51:17 +0100,
> > Matthieu Lemerre <address@hidden> wrote:
> >> Neither task nor deva was started for me. I recompiled a l4 kernel and
> >> it worked. (Maybe I was using the old patch for pistachio 0.2)
> >
> > If your grub config is right, and deva is not started, then you should
> > look at using the latest pistachio as required in the README.  deva
> > does have a cheesy kbd driver which echoes kbd input, and ESC will
> > trigger int $3 to get you into the debugger if you enabled the trap in
> > the configuration of pistachio.  That should give you a hint if it is
> > running.
> 
> I was in fact meaning that I managed to make it work.

And I was not reading carefully enough ;)  Great to hear it works, though.
 
> I looked at it, and I was wondering when work on the device driver
> framework could begin? I think it would require some work and I would
> be happy to contribute :) (but I would also be happy to contribute to
> any other thing)

A lot of work could begin right now, including some work of deva.
There is even already a module deva in the Hurd cvs repo, but it's not
much in there yet.

> > could try qemu btw, which works fine for me when I run it within a gdb
> > session (don't ask).
> 
> It works for me without using gdb, but I had to put some special arguments :
> 
>    qemu -user-net -std-vga -fda image -boot a
> (using a modified grub floppy disk image)

And you have your kernel and files on the floppy disk image?  I use
tun/tap and a grub boot disk which then uses tftpboot.  I need gdb to
make grub see the qemu network card.
 
> > Yes, it is missing.  Want to send in a patch?
> 
> Of course :), I will include it in my next patch.

Mmh, ok.  Usually we prefer separate patches for distinct issues, so
we can apply them individually.  But in this case they are related
enough to have them in one if you prefer it that way.
 
> >> -What is the bug in _L4_msg_get_string_item?
> >
> > Wish I could remember with certainty, but it may very well be that I
> > think that it possibly violates the strict aliasing rules of ISO C99.
> > Ie, the casts of &msg[] pointers to the string type has to be replaced
> > with explicit assignments to msg[] from .raw parts of the string item,
> > as I did for the other similar functions at some time.
> >
> > I don't know if you are familiar with the strict aliasing rules, if
> > the above sounds like gibberish to you, let me know and I will
> > elaborate.
> >
> Please do, I tried to find on google but I didn't find. I would be
> pleased to know.

Ok, the actual rules are complicated, but the basic rule is that two
pointers to different types never ever point to overlapping memory
regions.  This means that you can't cast pointers to other types and
expect them to work.  The compiler can just optimize away derefences
to pointers which have the value of a different type.  An exception is
char*, which can always be used (otherwise memcpy and friends would
not do what you expect).

For example, you can not allocate memory for a struct foo *, fill the
struct with data, then cast the pointer to a struct bar * and read it
out from that pointer.  The compiler may reorder and move the "reading
out via struct bar" part before the "writing to struct foo" part, and
then the data you have written to the struct foo will not be visible
through the struct bar * pointer.

This is even true if the two structs are totally identical, because
two structs with different names are by definition incompatible types,
irregardless of their actual representation, their layout, or
whatever.  The same goes for unions, and so on.

The bottom line is that as soon as you associate a memory region with
a type, you are stuck to that type, and that type only.  The only
exception is char *, which you can always use, and unions, where you
can do type punning between different members, but only if you do all
accesses through the union type.

So, for example, you can memcpy the memory pointed to by a struct foo
* to a memory pointed to by a struct bar *, and then access the data
through the struct bar * pointer.  A clever compiler could optimize
the memcpy() away, but of course nobody does that.

In fact, this means that a lot of code in libl4 still breaks the
aliasing rules, because I cast pointers to a type to a pointer to a
union type which has that type as a member.  This is not quite good
enough, as all accesses have to happen through the union type
(type-punning.  Note that even that is not official C99 but a gcc
extension!).  Bollocks.  I wrote that code before understanding the
aliasing rules thoroughly.

Watch out for code like this:

Example 1:

typedef _L4_word_t _L4_msg_t[_L4_NUM_MRS];
typedef union
{
  _L4_msg_t _msg;
  __L4_msg_tag_t tag;
} __L4_msg_t;

void
_L4_msg_get (_L4_msg_t msg, _L4_word_t *untyped, void *any_typed)
{
  __L4_msg_t *_msg = (__L4_msg_t *) msg;

...
}

Doesn't work, because _L4_msg_t and __L4_msg_t are NOT compatible!
Despite _L4_msg_t being the type of one of the union members.  For
type punning, you have to do all accesses through the union!  The
compiler can assume that msg and _msg are totally different pointers
to non-overlapping memory regions.

I use this a lot in the libl4 code, and in fact tried to avoid
aliasing problems this way, see what I knew!  This must be redone.
Maybe I have to expose the union type.

Example 2:

_L4_word_t
_L4_msg_get_string_item (_L4_msg_t msg, _L4_word_t nr,
                         _L4_string_item_t *string_item)
{
  [...]

  __L4_string_item_t *_string_item = (__L4_string_item_t *) string_item;

  [...]

     *((__L4_string_item_t *) &msg[pos]) = *_string_item;

  [...]
}


The same problem as above in the _string_item pointer creation, but it
goes on: I cast a word pointer to a string item pointer.  Doesn't work
at all.  Breaks aliasing rules.  So this examples actually breaks the
aliasing rules twice.

Now looking back at this code I see that I was horribly confused at
that time.

Don't believe you can get around these rules.  I spent at some point a
couple of days trying to find my ways around them, and it just doesn't
work.  Don't assume you can safely break them - all functions in libl4
are inlined, and you don't know how they are used.  I had libl4 string
item code break under me because of aliasing rules violation.  A whole
loop was simply optimized away by that.

Thanks,
Marcus