[Bug ld/20070] New: LLVM gold plugin(LLVMgold.so) report Unexpected resolution failure on ld when LTO, but pass on gold

[steven.shi at intel dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20070

            Bug ID: 20070
           Summary: LLVM gold plugin(LLVMgold.so) report Unexpected
                    resolution failure on ld when LTO, but pass on gold
           Product: binutils
           Version: 2.26
            Status: NEW
          Severity: critical
          Priority: P2
         Component: ld
          Assignee: unassigned at sourceware dot org
          Reporter: steven.shi at intel dot com
  Target Milestone: ---

Created attachment 9249
  --> https://sourceware.org/bugzilla/attachment.cgi?id=9249&action=edit
testcase to reproduce LLVMgold.so Unexpected resolution failure on ld

I'm enabling clang LTO feature to to improve code size of Uefi standard
(http://www.uefi.org/) firmware (https://github.com/tianocore/edk2). My project
is in https://github.com/shijunjing/edk2 branch llvm :
https://github.com/shijunjing/edk2/tree/llvm.

I find ld in binutils 2.26 cannot co-work correctly with LLVM gold plugin
(LLVMgold.so), and the LLVM gold plugin will report unexpected resolution
failure as below when ld do the optimization. But this failure will not happen
on gold.

Unexpected resolution
UNREACHABLE executed at
/home/jshi19/llvm-3.8.0.src/tools/gold/gold-plugin.cpp:679!


Below is the steps on how to setup clang compiler and reproduce the clang LTO
link failure:

1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g.
http://www.llvm.org/releases/3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz
and extract it as ~/clang38).
2. Decompress and rename the debug version LLVM gold plugin in
https://github.com/shijunjing/edk2/blob/llvm/BaseTools/Bin/LLVMgold-debug.tar.gz
as LLVMgold.so. Copy it to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so)
3. Copy GNU Binutils 2.26 linker ld to /usr/bin/ld
4. run below clang LTO link command:

~/clang38/bin/clang -o Hello.dll -flto -nostdlib -Wl,-n -Wl,-q
-Wl,--gc-sections -Wl,-z,common-page-size=0x40 -Wl,--entry,_ModuleEntryPoint
-Wl,-u,_ModuleEntryPoint -Wl,-Map,Hello.map -Wl,-melf_x86_64
-Wl,--oformat=elf64-x86-64 -Wl,--start-group,,@static_library_files.lst
-Wl,--end-group

ld 2.26 fail in gold plugin optimization with below output:
steven: getModuleForFile pass
Unexpected resolution
UNREACHABLE executed at
/home/jshi19/llvm-3.8.0.src/tools/gold/gold-plugin.cpp:679!
clang-3.8: error: unable to execute command: Aborted (core dumped)
clang-3.8: error: linker command failed due to signal (use -v to see
invocation)

5. Copy GNU Binutils 2.26 gold linker ld-new to /usr/bin/ld
6. re-run step 4 clang LTO link command again:

~/clang38/bin/clang -o Hello.dll -flto -nostdlib -Wl,-n -Wl,-q
-Wl,--gc-sections -Wl,-z,common-page-size=0x40 -Wl,--entry,_ModuleEntryPoint
-Wl,-u,_ModuleEntryPoint -Wl,-Map,Hello.map -Wl,-melf_x86_64
-Wl,--oformat=elf64-x86-64 -Wl,--start-group,,@static_library_files.lst
-Wl,--end-group


gold pass gold plugin optimization with below output:
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
steven: getModuleForFile pass
/usr/bin/ld: internal error in do_layout, at
../../binutils-2.26/gold/object.cc:1819
clang-3.8: error: linker command failed with exit code 1 (use -v to see
invocation)
Above gold do_layout issue is a known bug which is not related LLVM LTO, and
has been tracked in Bug 20062
https://sourceware.org/bugzilla/show_bug.cgi?id=20062

The "steven: getModuleForFile pass" is debug message I especially added in the
debug version LLVMgold.so, which is line 905 in
llvm-3.8.0.src\tools\gold\gold-plugin.cpp. I have a gold-plugin.cpp copy in
this folder.



This folder example libraries are abstracted from Uefi Hello module. You can
build them by yourself from https://github.com/shijunjing/edk2/tree/llvm

0. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g.
http://www.llvm.org/releases/3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz
and extract it as ~/clang38).
0. Decompress and rename the debug version LLVM gold plugin in
https://github.com/shijunjing/edk2/blob/llvm/BaseTools/Bin/LLVMgold-debug.tar.gz
as LLVMgold.so. Copy it to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so)
1. Setup EDK2 build environment as steps in the link:
https://github.com/tianocore/tianocore.github.io/wiki/Using-EDK-II-with-Native-GCC
 
2. git clone https://github.com/shijunjing/edk2 (e.g. ~/edk2)
3. $ cd edk2
4. $ git checkout llvm
5. $ export CLANG38_BIN=path/to/your/clang38/ (e.g. export
CLANG38_BIN=~/clang38/bin/)
6. $ source edksetup.sh
7. $ make -C BaseTools/Source/C
8. Comment out the line 33, 34 in ~/edk2/AppPkg/Applications/Hello/Hello.c,
which will trigger significant LTO optimization. I have a updated Hello.c copy
in this folder.
9. $ build -t CLANGLTO38 -a X64 -p AppPkg/AppPkg.dsc -m
AppPkg/Applications/Hello/Hello.inf

After build, you can find all intermediate files in below folder
~/edk2/Build/AppPkg/DEBUG_CLANGLTO38/X64/AppPkg/Applications/Hello/Hello

-- 
You are receiving this mail because:
You are on the CC list for the bug.