Re: [Lightning] Porting GNU Smalltalk to lightning 2

[Paulo César Pereira de Andrade]

2014-10-26 12:38 GMT-02:00 Holger Hans Peter Freyther <address@hidden>:
> On Sun, Oct 26, 2014 at 10:46:32AM -0200, Paulo César Pereira de Andrade 
> wrote:
>
>> > oh wow. In regard to the inline caches. Maybe you can think of a
>> > better way to invalidate them. You should invalidate your ICs from
>>
>>   Actually, I enabled it back in
>> https://github.com/pcpa/smalltalk/commit/c1cac9ab3f6abde6bc53127d4ca07428c77d1b21
>> I had the jump test reversed... I reversed the logic at some point when
>> doing some trial&error :(
>
> Think of something like the code below
>
> Object subclass: BaseClass [
>         myBaseMethod [
>                 self doStuff
>         ]
> ]
>
> BaseSubclass subclass: NewClass [
> ]
>
> inst := NewClass new.
> self myBaseMethod.
>
> NewClass extend [
>         myBaseMethod [
>                 self doStuffDifferently.
>         ]
> ]
>
>
> With the "self myBaseMethod" the inline cache will be initialized and
> BaseClass>>#myBaseMethod has been found. Then we install a new method
> in the case NewClass class and "Behavior" will ask the JIT to flush
> the caches as the look-up could have changed.
>
> When we are discarding a CompiledMethod. We want to make sure that
> no call-site refers to the soon to be dead code.

  Ok, I think :)

  I rewrote another portion of code to use the reverse hash to map a
native code pointer to a method_entry pointer, as it appears I had made
another incorrect assumption in the initial commit, see

https://github.com/pcpa/smalltalk/commit/d9b0ca4dff2c31f3f5072fc39756e3bc25864a1e

now it works exactly as the build with lightning 1, and will pass all
tests, and hang in the same test as the lightning 1 build.

The incorrect assumption was that a method_cache_entry nativeCode
pointer was immutable, what is obviously wrong, and to use a mmap 'ed
buffer, need the reverse hash. It is possible to tell lightning 2 where to
write jit to, so it could still use a variably sized method_entry structure,
but I prefer not, because that means mapping down to the page boundary
before and up to after it as executable...

  Now I do not recall any other "assumption" I made when reworking
the code to translate from lighting 1 to lightning 2. But the x86_64
problem is still there. What I think is a very likely culptrit is
libgst/xlat.c:analyze_dividend()

> What I was refering to was the question is you could think of a way
> to not have to walk all methods to invalidate the cache. Either by
> embedding an "age"/"version" that we compare and act on (and only
> walk all the methods in case the age overflows).

  That depends on how much more time I can invest on it, I am
afraid I am still quite new to this code. But in comparison, code
that does a lot more complex jit generation, but because I wrote
all bits, I find it very easy to debug, because I fully understand
everything...

>>   I looks quite better now, after this commit
>> https://github.com/pcpa/smalltalk/commit/6864d42b53bc34ce307a3d953125a985ba13d3d1
>> that should be what you are talking about. In my first reading of
>> the code I thought something like in the above commit was not
>> required. I mean, why does it need to adjust jit translations to
>> no longer reference translations that are about to be deleted?
>> If it references them, they are reachable and should not be
>> deleted... (Maybe the code was rewritten and need a new
>> translation?)
>
> I think the code is wrong for the "new method in the look-up chain"
> but I don't fully understand (as I didn't read the code to that
> detail) why the cache_ip needs a reverse look-up.
>
>>   How do I enabled a build type that calls gc at every possibility?
>> Some kind of gc debug build? I suspect it is failing some tests in
>> make check due to the gc "resetting" references to code about to
>> be deleted. The few now failing tests are passing in the build with
>> the bundled lightning 1, but I believe it may be due gc called a lot
>> more frequently.
>>   GC should be being called a lot more frequently due to the way
>> lightning 2 does translations, where it mmap's a code buffer, so,
>> there are a lot of "r-x" 4K buffers around, where lightning 1x would
>> generate as small as 100 bytes buffers (appended to variably sized
>> method_entry structures).
>>   BTW, it generates so much buffers where the difference is only
>> one embedded constant, it should be somehow parameterized.
>
>
> lightning: Well the mmap/malloc calls are not managed by the
> garbage collection. The CompiledMethod that holds a pointer to
> the native code is is managed by the GC.

  Ok. I do not know the GC details, so, was unsure if it could be checking
something else, like process allocated pages or something, but I
understand it does not directly know about the mmap'ed pages.

> kind regards
>         holger

Thanks,
Paulo