[Bug libctf/29242] New: ld crash when deduplicating CTF from multiple .o files with the same name

[nick.alcock at oracle dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29242

            Bug ID: 29242
           Summary: ld crash when deduplicating CTF from multiple .o files
                    with the same name
           Product: binutils
           Version: 2.38
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libctf
          Assignee: unassigned at sourceware dot org
          Reporter: nick.alcock at oracle dot com
  Target Milestone: ---

We observe on Gentoo a crash when linking Mesa 22.0.3 when compiled with -gctf.
GDB reports:

Starting program: /usr/bin/ld --eh-frame-hdr -m elf_x86_64 -shared -o
src/gallium/targets/dri/libgallium_dri.so
/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/../../../../lib64/crti.o
/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/crtbeginS.o -L/usr/lib/llvm/13/lib64
-L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64
-L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64
-L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64
-L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64 -L/usr/lib/llvm/13/lib64
-L/usr/lib/llvm/13/lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1
-L/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/../../../../lib64 -L/lib/../lib64
-L/usr/lib/../lib64
-L/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/../../../../x86_64-pc-linux-gnu/lib
-L/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/../../..
src/gallium/targets/dri/libgallium_dri.so.p/target.c.o
src/gallium/targets/dri/libgallium_dri.so.p/megadriver_stub.c.o --as-needed
--no-undefined --start-group -soname libgallium_dri.so -O1 --as-needed
--defsym=__gentoo_check_ldflags__=0 -rpath /../../../mapi/shared-glapi
-rpath-link
/var/tmp/portage/media-libs/mesa-22.0.3/work/mesa-22.0.3-abi_x86_64.amd64/src/mapi/shared-glapi
src/gallium/frontends/dri/libdri.a src/util/libmesa_util.a
src/util/format/libmesa_format.a src/mesa/libmesa.a src/compiler/glsl/libglsl.a
src/compiler/glsl/glcpp/libglcpp.a src/compiler/nir/libnir.a
src/compiler/libcompiler.a src/mesa/libmesa_sse41.a
src/gallium/auxiliary/libgalliumvl.a src/gallium/auxiliary/libgallium.a
src/mapi/shared-glapi/libglapi.so.0.0.0
src/gallium/auxiliary/pipe-loader/libpipe_loader_static.a
src/loader/libloader.a src/util/libxmlconfig.a
src/gallium/winsys/sw/null/libws_null.a src/gallium/winsys/sw/wrapper/libwsw.a
src/gallium/winsys/sw/dri/libswdri.a
src/gallium/winsys/sw/kms-dri/libswkmsdri.a
src/gallium/drivers/llvmpipe/libllvmpipe.a
src/gallium/drivers/softpipe/libsoftpipe.a src/gallium/drivers/r300/libr300.a
src/gallium/winsys/radeon/drm/libradeonwinsys.a
src/gallium/drivers/r600/libr600.a
src/gallium/drivers/radeonsi/libradeonsi_gfx6.a
src/gallium/drivers/radeonsi/libradeonsi_gfx7.a
src/gallium/drivers/radeonsi/libradeonsi_gfx8.a
src/gallium/drivers/radeonsi/libradeonsi_gfx9.a
src/gallium/drivers/radeonsi/libradeonsi_gfx10.a
src/gallium/drivers/radeonsi/libradeonsi_gfx103.a
src/gallium/drivers/radeonsi/libradeonsi.a
src/gallium/winsys/amdgpu/drm/libamdgpuwinsys.a src/amd/addrlib/libaddrlib.a
src/amd/common/libamd_common.a src/amd/llvm/libamd_common_llvm.a
src/gallium/winsys/nouveau/drm/libnouveauwinsys.a
src/gallium/drivers/nouveau/libnouveau.a src/gallium/drivers/i915/libi915.a
src/gallium/winsys/i915/drm/libi915drm.a src/gallium/drivers/iris/libiris.a
src/gallium/drivers/iris/libiris_per_hw_ver80.a
src/gallium/drivers/iris/libiris_per_hw_ver90.a
src/gallium/drivers/iris/libiris_per_hw_ver110.a
src/gallium/drivers/iris/libiris_per_hw_ver120.a
src/gallium/drivers/iris/libiris_per_hw_ver125.a
src/intel/compiler/libintel_compiler.a src/intel/dev/libintel_dev.a
src/intel/isl/libisl.a src/intel/isl/libisl_per_hw_ver40.a
src/intel/isl/libisl_per_hw_ver50.a src/intel/isl/libisl_per_hw_ver60.a
src/intel/isl/libisl_per_hw_ver70.a src/intel/isl/libisl_per_hw_ver75.a
src/intel/isl/libisl_per_hw_ver80.a src/intel/isl/libisl_per_hw_ver90.a
src/intel/isl/libisl_per_hw_ver110.a src/intel/isl/libisl_per_hw_ver120.a
src/intel/isl/libisl_per_hw_ver125.a src/intel/isl/libisl_tiled_memcpy.a
src/intel/isl/libisl_tiled_memcpy_sse41.a src/intel/blorp/libblorp.a
src/intel/perf/libintel_perf.a src/intel/common/libintel_common.a
src/intel/ds/libintel-driver-ds.a src/gallium/winsys/iris/drm/libiriswinsys.a
src/gallium/drivers/crocus/libcrocus.a
src/gallium/drivers/crocus/libcrocus_per_hw_ver40.a
src/gallium/drivers/crocus/libcrocus_per_hw_ver45.a
src/gallium/drivers/crocus/libcrocus_per_hw_ver50.a
src/gallium/drivers/crocus/libcrocus_per_hw_ver60.a
src/gallium/drivers/crocus/libcrocus_per_hw_ver70.a
src/gallium/drivers/crocus/libcrocus_per_hw_ver75.a
src/gallium/drivers/crocus/libcrocus_per_hw_ver80.a
src/gallium/winsys/crocus/drm/libcrocuswinsys.a --build-id=sha1 --gc-sections
--version-script
/var/tmp/portage/media-libs/mesa-22.0.3/work/mesa-22.0.3/src/gallium/targets/dri/dri.sym
--dynamic-list
/var/tmp/portage/media-libs/mesa-22.0.3/work/mesa-22.0.3/src/gallium/targets/dri/../dri-vdpau.dyn
/usr/lib64/libdrm.so -lLLVM-13 /usr/lib64/libexpat.so /usr/lib64/libz.so
/usr/lib64/libzstd.so -lLLVM-13 -lLLVM-13 /usr/lib64/libdrm_radeon.so -lLLVM-13
/usr/lib64/libelf.so -lLLVM-13 -lLLVM-13 -lLLVM-13 -lLLVM-13 -lLLVM-13
-lLLVM-13 -lLLVM-13 -lLLVM-13 -lLLVM-13 /usr/lib64/libdrm_amdgpu.so -lLLVM-13
/usr/lib64/libdrm_nouveau.so /usr/lib64/libdrm_intel.so --end-group -lstdc++
-lm -lgcc_s -lpthread -lc -lgcc_s
/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/crtendS.o
/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/../../../../lib64/crtn.o
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f71090 in htab_hash_string () from
/usr/lib64/binutils/x86_64-pc-linux-gnu/9999/libbfd-2.38.50.20220428.gentoo-sys-devel-binutils-st.so
(gdb) bt
#0  0x00007ffff7f71090 in htab_hash_string () from
/usr/lib64/binutils/x86_64-pc-linux-gnu/9999/libbfd-2.38.50.20220428.gentoo-sys-devel-binutils-st.so
#1  0x00007ffff7f70f31 in htab_remove_elt () from
/usr/lib64/binutils/x86_64-pc-linux-gnu/9999/libbfd-2.38.50.20220428.gentoo-sys-devel-binutils-st.so
#2  0x00007ffff7e63075 in ctf_dynhash_remove () from
/usr/lib64/binutils/x86_64-pc-linux-gnu/9999/libctf.so.0
#3  0x00007ffff7e6a432 in ctf_link () from
/usr/lib64/binutils/x86_64-pc-linux-gnu/9999/libctf.so.0
#4  0x0000555555574706 in lang_process ()
#5  0x00005555555605ae in main ()

This is a bug in the error-handling code: we are double-freeing a ctf_dict_t
when an error happens in symbol emissions. Upon fixing that, the underlying bug
reveals itself. The symptoms look very similar but they are followed by a lot
of suspicious messages of the general form

CTF warning: type 5d0 for symbol GFX8_MI_COPY_MEM_MEM_pack in input file
/var/tmp/portage/media-libs/mesa-22.0.3/work/mesa-22.0.3-abi_x86_64.amd64/../mesa-22.0.3/src/gallium/drivers/iris/iris_query.c
not found: skipped

(etc), followed by an even more suspicious

CTF error: symbol isl_encode_aux_mode in input file
/var/tmp/portage/media-libs/mesa-22.0.3/work/mesa-22.0.3-abi_x86_64.amd64/../mesa-22.0.3/src/intel/isl/isl_surface_state.c
found conflicting even when trying in per-CU dict.

This should absolutely never happen: output per-CU dicts are associated 1:1
with specific input dicts, and in single input dicts any given type must have a
single unambiguous definition and cannot possibly be conflicting.

The underlying cause is that Mesa compiles the same object files many times
with different #defines, stuffs them in the libiris_per_hw*.a libraries shown
above, then links them all together. ctf-link stores CTF dicts in maps keyed
not by library name but by input CU name, and if a given input has been built
repeatedly with different flags all those inputs will have the same CU name,
even though they have different object files and may have different types in
them!

Clearly this should not fail, even if it's not possible to make the result
especially easy to understand (but then, the underlying problem is weird: no
debugger is going to distinguish between "type foo from x.c compiled with -DFOO
and type foo from x.c compiled with -DBAR": but both types should be
represented somewhere and the linker shouldn't crash).

-- 
You are receiving this mail because:
You are on the CC list for the bug.