GNUstep on Windows using Clang + MSVC ABI

[Frederik Seiffert]

Good news everyone: I’ve successfully built Base on Windows with Clang, 
libobjc2, and the MSVC ABI. This results in lib, DLL, and PDB files that should 
be usable in any Windows app without using the MinGW toolchain.

I’ve opened the following pull requests with the required changes:

- tools-make: https://github.com/gnustep/tools-make/pull/14
- libs-base: https://github.com/gnustep/libs-base/pull/168

A couple of notes:


- Linking subproject object files directly (instead of merging)

As David had suspected, merging the subproject object files turned out to be an 
issue: `ld -r` is not available on Windows, and using `ar cr` resulted in a 
subproject.o that didn’t contain all the symbols (I’m guessing because ar 
doesn't correctly handle PE/COFF files).

So I looked into how we could instead use the subproject object files directly 
when linking the project, and came up with this solution: instead of merging 
the object files into subproject.o, we simply write the list of object files to 
a subproject.txt file (in the same location), and read that to create 
SUBPROJECT_OBJ_FILES.

The change is pretty minimal and seems to work fine on all 
platforms/configurations in CI:
https://github.com/gnustep/tools-make/commit/434f957df0ad81b52a09e3f8c4a200734898b342

Given that this affects all platforms I’d appreciate everyone’s feedback on 
this, but given that issues with incremental linking with various setups has 
been talked about multiple times on the mailing list, I hope that this change 
will be an overall improvement.


- Building

Building with this setup requires using a standard (non-MinGW) Clang that e.g. 
comes with Visual Studio or is available as pre-built binary from the LLVM 
website, and requires passing a host to configure like 
--host=x86_64-pc-windows. Invoking `clang -v` should show a target like 
"x86_64-pc-windows-msvc". It *might* be that using a MinGW Clang works too when 
invoked with --target x86_64-pc-windows-msvc, but I haven’t tried that.

The build is best done in an MSYS2 shell that does not have any additional 
*-devel packages installed that might get picked up by configure. Alternatively 
--disable-xxx flags can be used to prevent these dependencies to be picked up.

I’m currently building like this:

# tools-make
export CC=/C/LLVM/bin/clang
export CXX=/C/LLVM/bin/clang++
export OBJCXX=/C/LLVM/bin/clang++
./configure --host=x86_64-pc-windows --with-library-combo=ng-gnu-gnu 
--with-runtime-abi=gnustep-2.0 --prefix=/c/GNUstep/x64 LDFLAGS="-fuse-ld=lld"
make install

# libs-base
. /c/GNUstep/x64/share/GNUstep/Makefiles/GNUstep.sh
./configure --host=x86_64-pc-windows --disable-iconv --disable-tls 
--disable-icu --disable-xml
make -j12 install


- Dependencies

While most dependencies should be available via NuGet, I have not been able to 
figure out yet how we might integrate NuGet packages into the build process. 
Downloading the packages results in individual folders per package with deeply 
nested folder structures containing the libraries for different architectures, 
debug/release targets, and Visual Studio versions.

For now I have manually copied the headers and libffi.lib import library 
(renamed to ffi.lib) from the libffi package, and the libffi.dll from the 
libffi.redist package.

For pthreads I have built http://www.sourceware.org/pthreads-win32/ from source 
by invoking "nmake clean VC" in a VS native tools command prompt, and manually 
copied the headers (pthread.h, sched.h, semaphore.h), pthreadVC2.lib (renamed 
to pthread.lib), and phtreadVC2.dll.

I haven’t build with any other dependencies so far but that will be my next 
endeavor. If anyone has suggestions on how the NuGet packages might be used 
directly I’d appreciate any input. This is also the main reason I haven’t set 
this up on CI yet, as we’d have to replicate the currently manual process 
somehow.

I’m also thinking about writing some scripts similar to tools-android that 
could build all the dependencies and GNUstep itself, ideally also for the 
different architectures and debug/release targets.


- NSFileManager support

NSFileManager is currently non-functional with this setup due to dirent.h 
missing (MinGW has it, but Windows doesn’t).

I found this MIT-licensed header-only implementation – would that be ok to add 
to the project?
https://github.com/win32ports/dirent_h/blob/master/dirent.h


- Autoconf GNUC / GCC / Clang detection

When targeting the MSVC ABI, Clang does not define `__GNUC__`, which causes 
Autoconf to not define $GCC (i.e. "checking whether we are using the GNU C 
compiler" will be NO). I thus removed $GCC as a pre-requisite in 
GS_CHECK_CC_IS_CLANG(), so that Clang is still correctly detected in this setup.


- Debug/Release CRT

Just a heads up that on Windows the C runtime libraries (CRT) come in two 
versions for debug and release builds (e.g. msvcrt and msvcrtd). The libraries 
that are used seem to have to match between all DLLs – e.g. building libobjc2 
for debug will cause all sorts of crashes when Base is build for release, e.g. 
when freeing memory allocated by libobjc2 in Base.


- Linker issues

I tried using the MS linker (link.exe), but while that seems to generally work 
fine linking ObjC files, it throws up at some configure tests (specifically 
when testing "whether objc really works"). Using LLD works though, so one must 
run Make configure with LDFLAGS="-fuse-ld=lld".

I also had to bypass the config checks for objc_sync_enter(), 
objc_setProperty(), _Block_copy(), and non-fragile-abi support and assume these 
functions are there, as these checks result in the following linker errors.

        lld-link: error: relocation against symbol in discarded section: 
__start_.objcrt$SEL
        >>> referenced by C:\msys64\tmp\conftest-78a937.o:(.objc_init)

This is reproducible when building a file that calls ObjC runtime functions but 
doesn’t actually contain any ObjC code. Not sure if that’s expected behavior or 
bug.

While 64-bit builds works fine with this setup, I have not been able to get 
32-bit builds to work yet due to this linker error, but I’m probably just 
missing some flags somewhere:

        lld-link: error: libcmt.lib: machine type x64 conflicts with x86

Also, as David noted, this setup requires dllexport/dllimport on all ObjC 
classes, so I annotated all our ObjC class interfaces with GS_EXPORT_CLASS (and 
also added missing GS_DECLARE annotations in externs.m), in order for these 
symbols to be correctly exported in the DLL.


Looking forward to any feedback.

Thanks,
Frederik