|
| From: | Karl Yerkes |
| Subject: | Re: [Tinycc-devel] Two compilers at once |
| Date: | Fri, 21 Jun 2019 15:09:57 -0700 |
On Fri, Jun 21, 2019 at 4:58 AM Karl Yerkes <address@hidden> wrote:
>
>
> Steve Fan --
>
> On Thu, Jun 20, 2019 at 7:14 PM Steve Fan <address@hidden> wrote:
>>
>> I’m sorry to intervene in the matter, but as you look at the design of TCC, you will see that it is not designed for re-entrancy.
>
>
> yes. i understand. i am not trying to make it reentrant. i want to hot-recompile-swap a function with very low latency and i am willing to settle for a hack.
>
>
>> Foolish younger me, I used to have a hardworking re-entrant fork of TCC working but the one of the project owner denounced my idea, calling me to adapt TLS for each and every global variables instead of rewriting most of the TCC functions to add TCCState for them.
>
>
> good for you.. and that sucks.
>
>
>> TCC is just a hobbyist bone picked from OTCC by the legend Fabrice Bellard himself, but he has abandoned his child, TCC, ever since.
>
>
> yes. i get that.
>
>
>> I used to be like you, I want to research on TCC, but now I do warn you to not touch this egregious code again.
>
>
> i am not aiming to change the TCC source code. i actually only want to hack around what currently exists. that said, i might end up helping out somehow.
>
>
>> Go figure out your own parser. Maybe learn Bison or ANTLR for a basic parser, then read some paper about register allocation (which is a very bitchy problem for it was actually NP-Complete), read the specs for ELF, and read the respective assembly handbook for each and every codegen backend. You can reuse the ELF/PE code from TCC I reckon, but
>
>
> thank you. this is good advice and good info. i know parsing and grammars. i write my grammars/parsers in perl6 :)
>
> maybe generating x86, ARM, or LLVM-IR is not that hard and i should go do that. (right now it is black magic to me.)
>
> but my aim is actually to hack something together to further my aims in a different domain: sound.
>
> i am quite close with TCC.
>
>
>>
>> just don’t try to make anything happen to TCC, a dying project.
>
>
> i hear your warning. you may very well be right, but to me TCC seems... mature, not dying. maybe it's not going to be much more than what it is, but i bet it'll hold its value for a while.
>
> -- karl
>
>>
>>
>>
>> From: Karl Yerkes
>> Sent: Friday, 21 June 2019 9:24 AM
>> To: address@hidden
>> Subject: Re: [Tinycc-devel] Two compilers at once
>>
>>
>>
>> thank you for your replies. your help and insights are much appreciated.
>>
>>
>>
>> I'm loving TCC!
>>
>>
>>
>> however, i've reached the limit of my current hot-swap function hack that uses two state-commingling TCC instances. after a while, the instances stop working properly and I get strange compile errors.
>>
>>
>>
>> so, i am interested in executing some hack to allow multiple independent TCC instances in the same application. i am certain that "the old trick of just opening a pipe, forking and
>>
>> then sending back the result" will not work for my purposes. if i can't get one of these hacks working, i have to go look at LLVM and cling again :/
>>
>>
>>
>> as i understand it, option 2 (from Giovanni Mascellani) is using libtcc to compile libtcc from source into memory. that sounds awesome. i spent most of today trying this on windows.. and also learning to compile TCC on windows. here's my attempt:
>>
>>
>>
>> // boot.cpp
>> //
>> // tcc version 0.9.27 (x86_64 Windows)
>> // install: c:/tcc
>> // include:
>> // c:/tcc/include
>> // c:/tcc/include/winapi
>> // libraries:
>> // c:/tcc/lib
>> // C:/Windows/system32
>> // libtcc1:
>> // c:/tcc/lib/libtcc1-64.a
>> //
>> // build and run with:
>> // cd tinycc
>> // cl /MD boot.cpp c:\tcc\libtcc64.lib
>> // boot.exe
>> //
>> // fails like this:
>> // need 457827 bytes
>> // relocating to 21f0a402fd0
>> // tcc: error: library 'libtcc1-64.a' not found
>> // Assertion failed: dec != nullptr, file boot.cpp, line 69
>> //
>> #include "libtcc.h"
>>
>> #include <cassert>
>> #include <iostream>
>>
>> int main() {
>> TCCState* instance = tcc_new();
>>
>> // i'm on Windows 10 x64. what else do i need here?
>> tcc_set_options(instance, "-shared");
>> tcc_define_symbol(instance, "TCC_TARGET_X86_64", "");
>> tcc_define_symbol(instance, "TCC_TARGET_PE", "");
>> tcc_define_symbol(instance, "LIBTCC_AS_DLL", "");
>>
>> // i needed this define on Windows to get tcc_relocate in the DLL
>> tcc_define_symbol(instance, "TCC_IS_NATIVE", "");
>>
>> tcc_set_output_type(instance, TCC_OUTPUT_MEMORY);
>>
>> // maybe add more source files?
>> tcc_add_file(instance, "libtcc.c");
>>
>> int size = tcc_relocate(instance, (void*)0);
>> void* memory = malloc(size);
>>
>> printf("need %d bytes\n", size);
>> printf("relocating to %llx\n", (unsigned long long int)memory);
>> tcc_relocate(instance, memory);
>>
>> // TCCState *tcc_new(void);
>> using New = TCCState* (*)(void);
>> New _new = (New)tcc_get_symbol(instance, "tcc_new");
>> assert(_new != nullptr);
>>
>> // int tcc_set_output_type(TCCState *s, int output_type);
>> using SetOutputType = int (*)(TCCState*, int);
>> SetOutputType _set_output_type =
>> (SetOutputType)tcc_get_symbol(instance, "tcc_set_output_type");
>> assert(_set_output_type != nullptr);
>>
>> // int tcc_compile_string(TCCState *s, const char *buf);
>> using CompileString = int (*)(TCCState*, const char*);
>> CompileString _compile_string =
>> (CompileString)tcc_get_symbol(instance, "tcc_compile_string");
>> assert(_compile_string != nullptr);
>>
>> // int tcc_relocate(TCCState *s1, void *ptr);
>> using Relocate = int (*)(TCCState*, void*);
>> Relocate _relocate = (Relocate)tcc_get_symbol(instance, "tcc_relocate");
>> assert(_relocate != nullptr);
>>
>> // void *tcc_get_symbol(TCCState *s, const char *name);
>> using GetSymbol = void* (*)(TCCState*, const char*);
>> GetSymbol _get_symbol = (GetSymbol)tcc_get_symbol(instance, "tcc_get_symbol");
>> assert(_get_symbol != nullptr);
>>
>> // can i delete the instance now??
>> // my experiments suggest no.
>> // tcc_delete(instance);
>>
>> // use the new, relocated instance to compile something
>> TCCState* born_in_memory = _new();
>> assert(born_in_memory != nullptr);
>> _set_output_type(born_in_memory, TCC_OUTPUT_MEMORY);
>> _compile_string(born_in_memory, "int dec(int t) { return t - 1; }");
>> _relocate(born_in_memory, TCC_RELOCATE_AUTO);
>> using Foo = int (*)(int);
>> Foo dec = (Foo)_get_symbol(born_in_memory, "dec");
>> assert(dec != nullptr);
>> printf("dec(2) == %d\b", dec(2));
>> }
>>
>>
>>
>> am i on the right track here? any suggestions would help a lot. i think i could add the path to libtcc1-64.a, but i would rather compile all the .c files i need so there's no need to find libtcc1-64.a at runtime.
>>
>>
>>
>> -
>>
>>
>>
>> option 1 (from Christian Jullien) is something that i don't quite understand. i am just failing to connect the dots given i the pseudocode:
>>
>>
>>
>> // tcc1.cpp:
>> namespace tcc1 {
>> #include "libtcc.h"
>>
>> // does this mean i should compile libtcc as c++ (not c) under the tcc1 namespace?
>> };
>>
>> // tcc2.cpp:
>> namespace tcc2 {
>> #include "libtcc.h"
>>
>> // does this mean i should compile libtcc as c++ (not c) AGAIN under the tcc2 namespace?
>>
>> // OR does this mean i should write a wrapper interface for libtcc here?
>> };
>>
>> // main.cpp:
>> int main() {
>> tcc1::TCCState* a = nullptr;
>> char (*A)(int) = nullptr;
>> assert((a = tcc1::tcc_new()) != nullptr);
>> tcc2::TCCState* b =
>> tcc2::tcc_new(); // remove this line to make the program work
>> }
>>
>> also "But it may not be so simple. For some projects it worked flawlessly and failed for some others" is mysterious to me. my apologies for being too ignorant to put it together! a little more help on this option might help me a lot.
>>
>>
>>
>> sorry for the long email.
>>
>>
>>
>> -- karl
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Sat, May 25, 2019 at 12:30 AM Giovanni Mascellani <address@hidden> wrote:
>>
>> Hi,
>>
>> Il 25/05/19 06:24, Christian Jullien ha scritto:
>> > Because Karl uses C++11, it may be /theoretically/ possible for him to
>> > load two different tcc instances having different C++ namespaces and
>> > compiled in two distinct translation units.
>>
>> If you don't mind some quite dirty hacking (and if you mind it you
>> shouldn't probably use tcc), I also believe that you can have two
>> instances in the same address space by using the first one to compile
>> another one (or many others) and relocating them to different addresses.
>> Then each of them is a completely independent compiler with independent
>> state. Of course this means that you need tcc's source code at runtime.
>> Otherwise there is the old trick of just opening a pipe, forking and
>> then sending back the result.
>>
>> Have fun, Giovanni.
>> --
>> Giovanni Mascellani <address@hidden>
>> Postdoc researcher - Université Libre de Bruxelles
>>
>> _______________________________________________
>> Tinycc-devel mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/tinycc-devel
>>
>>
>>
>> _______________________________________________
>> Tinycc-devel mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/tinycc-devel
>
> _______________________________________________
> Tinycc-devel mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Stupide question, but can't you just use 2 instances of tcc in 2
diferent process ?
Maybe a simpler hack, instead of making it reentrent would be to add
optionaly _Thread_local
attribute to tcc globals and execute the 2 instances in 2 diferent threads.
Matthias,
_______________________________________________
Tinycc-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/tinycc-devel
| [Prev in Thread] | Current Thread | [Next in Thread] |