Compiling with -no-crt on Linux causes a segmentation violation

[amarz45]

Context

Compiling any Odin program with the no-crt -default-to-panic-allocator build flags on Linux causes that program to give a segmentation violation.

odin report output:

	Odin:    dev-2024-12
	OS:      Alpine Linux v3.21, Linux 6.12.2-0-lts
	CPU:     11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
	RAM:     7607 MiB
	Backend: LLVM 19.1.4

Expected Behavior

For the following program:

package example

import "core:fmt"

main :: proc() {
	fmt.println("lorem ipsum")
}

It is expected that it prints lorem ipsum when building with odin build . -no-crt -default-to-panic-allocator.

Current Behavior

segmentation violation

[flysand7]

Identified the source of the issue.

• base/runtime/core.odin:755: Initializes the default temp allocator as a reference to global_default_temp_allocator_data.
• base/runtime/core_builtin.odin:52: This variable is thread local, meaning it's accessed via the FS segment register, which is supposed to be initialized by the CRT but isn't, since -no-crt is used.

I'll suggest a quick workaround: Compiling your program with -no-thread-local makes the program run normally. This flag makes it so that @(thread_local) attribute is ignored. But note that Odin isn't designed to be used with multithreaded applications.

The other option is to compile with -default-to-nil-allocator instead, since this disables the initialization of the temp allocator and bypasses thread local access.

[flysand7]

I'll proceed with working on solidifying the stance on the thread locals in Linux w/o libc. I think this should be the correct set of changes:

1. Adding an error message when -thread-local is used in the presence of -no-crt on Unix platforms.
2. Explicitly requesting users to enable either -no-thread-local or -no-default-temp-allocator when compiling on Unix with -no-crt.
3. Handling -no-thread-local gracefully in the core library. This will probably involve renaming this flag to -no-threads and making core:sync and core:threads panic in the presence of the flag. Other parts of the core library that depend on these (if there are these parts), should allow usage in single-threaded code.

[Waqar144]

@flysand7 it still crashes. Tried the following program:

package main
main :: proc() {}

Compiled/run with:

odin run a.odin -file -no-crt -default-to-nil-allocator -no-thread-local -debug

Seems like it doesn't even get to _start.

(gdb) bt
#0  0x00007ffff7fe6b95 in ?? () from /lib64/ld-linux-x86-64.so.2
#1  0x00007ffff7fe3786 in ?? () from /lib64/ld-linux-x86-64.so.2
#2  0x00007ffff7fe50de in ?? () from /lib64/ld-linux-x86-64.so.2
#3  0x00007ffff7fe3dc8 in ?? () from /lib64/ld-linux-x86-64.so.2

Explicitly requesting users to enable either -no-thread-local or -no-default-temp-allocator when compiling on Unix with -no-crt.

You mean -default-to-nil-allocator? since there is no -no-default-temp-allocator option afaics

[flysand7]

Have you perhaps forgotten to build Odin?

$ git checkout master
$ git pull
$ ./build_odin.sh release
$ odin run examples/bug -no-crt -no-thread-local -default-to-nil-allocator
lorem ipsum

When ran with the example in the issue.

[Waqar144]

Yes, this is with latest:

waqar ~/test $ ~/projects/Odin/odin version
/home/waqar/projects/Odin/odin version dev-2025-01:223970671
waqar ~/test $ ~/projects/Odin/odin run a.odin -file -debug -no-crt -no-thread-local -default-to-nil-allocator
Segmentation fault (core dumped)
waqar ~/test $ ./a
Segmentation fault (core dumped)

[Waqar144]

The weird thing is #0 0x00007ffff7fe6b95 in ?? () from /lib64/ld-linux-x86-64.so.2. This should not be involved at all since the binary is not dynamic IIUC.

ldd ./a
not a dynamic executable

For example, with an exe built with zig:

gdb ./zig-out/bin/file
Reading symbols from ./zig-out/bin/file...
(gdb) starti
Starting program: /home/waqar/projects/file/zig-out/bin/file 

Program stopped.
start._start () at /home/waqar/bin/zig-linux/lib/std/start.zig:266
266         asm volatile (switch (native_arch) {

[flysand7]

lib64/ld-linux-x86-64.so.2

It is involved because you are running an executable. This. so file the dynamic loader, i. e. the thing that is launched by the kernel directly when you run the program via the execve syscall, and it's responsible for loading your executable into the memory and doing some things like resolving dynamic relocations, pre-loading libraries and providing a dlopen interface and a few other things. What is interesting though, is that you're crashing inside your dynamic loader. This should not happen at all.

Oh. Also, I have just realized that you're running alpine linux, which uses musl which is different from glibc that I tested with on Ubuntu. I don't have an alpine machine, but there are a few things I'm curious about. Can you run this again in gdb and show assembly around the point where it crashes? Ideally I'd also like to see the source code, I'm not sure how much effort you're willing to put to install debug versions of the dynamic linker.

[Waqar144]

I am using manjaro with glibc 2.40:

/usr/lib/libc.so.6 
GNU C Library (GNU libc) stable release version 2.40.

I did some quick debugging and I think its because odin doesn't tell the linker to omit dynamic linker. See:

readelf -p .interp ./a

String dump of section '.interp':
  [     0]  /lib64/ld-linux-x86-64.so.2

For zig, this section doesn't exist.

So for testing I made the following change:

diff --git a/src/linker.cpp b/src/linker.cpp
index 261d6e7a4..cf148ff9c 100644
--- a/src/linker.cpp
+++ b/src/linker.cpp
@@ -595,6 +595,7 @@ gb_internal i32 linker_stage(LinkerData *gen) {
 
                        if (build_context.no_crt) {
                                link_settings = gb_string_append_fmt(link_settings, "-nostdlib ");
+                               link_settings = gb_string_append_fmt(link_settings, "-Wl,--no-dynamic-linker ");
                        }
 
                        if (build_context.build_mode == BuildMode_StaticLibrary) {

and it works now. There is no interp section anymore. GDB takes me to _start as the first instruction.

This might all be totally bogus though.

[Waqar144]

Backtrace with symbols:

audit_list_add_dynamic_tag (list=0x7fffffffd6e0, main_map=0x7ffff7ffe2e0, tag=1879047932) at rtld.c:225                                                                                                  
225       const char *strtab = (const char *) D_PTR (main_map, l_info[DT_STRTAB]);
(gdb) bt
#0  audit_list_add_dynamic_tag (list=0x7fffffffd6e0, main_map=0x7ffff7ffe2e0, tag=1879047932) at rtld.c:225
#1  dl_main (phdr=<optimized out>, phnum=<optimized out>, user_entry=<optimized out>, auxv=<optimized out>) at rtld.c:1786
#2  0x00007ffff7fe3786 in _dl_sysdep_start (start_argptr=start_argptr@entry=0x7fffffffd9f0, dl_main=dl_main@entry=0x7ffff7fe53e0 <dl_main>) at ../sysdeps/unix/sysv/linux/dl-sysdep.c:141
#3  0x00007ffff7fe50de in _dl_start_final (arg=0x7fffffffd9f0) at rtld.c:494
#4  _dl_start (arg=0x7fffffffd9f0) at rtld.c:581
#5  0x00007ffff7fe3dc8 in _start () from /lib64/ld-linux-x86-64.so.2
#6  0x0000000000000001 in ?? ()
#7  0x00007fffffffde29 in ?? ()
#8  0x0000000000000000 in ?? ()

Full backtrace

(gdb) bt full
#0  audit_list_add_dynamic_tag (list=0x7fffffffd6e0, main_map=0x7ffff7ffe2e0, tag=1879047932) at rtld.c:225
        info = 0x0
        strtab = <optimized out>
        info = <optimized out>
        strtab = <optimized out>
#1  dl_main (phdr=<optimized out>, phnum=<optimized out>, user_entry=<optimized out>, auxv=<optimized out>) at rtld.c:1786
        main_map = 0x7ffff7ffe2e0
        file_size = 140737353896488
        file = <optimized out>
        i = <optimized out>
        rtld_is_main = <optimized out>
        tcbp = 0x0
        skip_env = <optimized out>
        state = {audit_list = {audit_strings = {0x34000000340 <error: Cannot access memory at address 0x34000000340>, 0x34000000340 <error: Cannot access memory at address 0x34000000340>, 
              0x34000000340 <error: Cannot access memory at address 0x34000000340>, 0x34000000340 <error: Cannot access memory at address 0x34000000340>, 
              0x34000000340 <error: Cannot access memory at address 0x34000000340>, 0x34000000340 <error: Cannot access memory at address 0x34000000340>, 
              0x34000000340 <error: Cannot access memory at address 0x34000000340>, 0x34000000340 <error: Cannot access memory at address 0x34000000340>, 0x0, 
              0x100 <error: Cannot access memory at address 0x100>, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, length = 0, current_index = 0, current_tail = 0x0, 
            fname = '\000' <repeats 40 times>, "\177\003\000\000\000\000\000\000W\367\f\t", '\000' <repeats 12 times>, "p\331\377\377\377\177\000\000\000p\374\367\377\177\000\000\004\000\000\000\000\000\000\000\340\330\377\377\377\177\000\000r\337\375\367\377\177\000\000\002\000\000\000\000\000\000\000\034\000\000\000\000\000\000\000 ", '\000' <repeats 15 times>, "\027\000\000\000\000\000\000\000\000\000\b\000\000\000\000\000@\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000@\000\000\000\000\000\000\000\377\377\377\377\377\377\377\377\000\0000\000\000\000\000\000\377\377\377\377\377\377\377\377"...}, library_path = 0x0, library_path_source = 0x0, preloadlist = 0x0, preloadarg = 0x0, glibc_hwcaps_prepend = 0x0, glibc_hwcaps_mask = 0x0, mode = rtld_mode_normal, 
          mode_trace_program = false, version_info = false}
        ld_so_name = <optimized out>
        __PRETTY_FUNCTION__ = "dl_main"
        has_interp = <optimized out>
        first_preload = 0x7ffff7ffe8d8
        r = 0x7ffff7ffe128 <_r_debug>
        rtld_ehdr = <optimized out>
        rtld_phdr = <optimized out>
        cnt = <optimized out>
        need_security_init = <optimized out>
        count_modids = <optimized out>
        preloads = <optimized out>
        npreloads = <optimized out>
        preload_file = "/etc/ld.so.preload"
        rtld_multiple_ref = <optimized out>
        was_tls_init_tp_called = <optimized out>
        consider_profiling = <optimized out>
        start = <optimized out>
#2  0x00007ffff7fe3786 in _dl_sysdep_start (start_argptr=start_argptr@entry=0x7fffffffd9f0, dl_main=dl_main@entry=0x7ffff7fe53e0 <dl_main>) at ../sysdeps/unix/sysv/linux/dl-sysdep.c:141
        dl_main_args = {phdr = 0x200040, phnum = 8, user_entry = 2350240}
#3  0x00007ffff7fe50de in _dl_start_final (arg=0x7fffffffd9f0) at rtld.c:494
        start_addr = <optimized out>
        start_addr = <optimized out>
        rtld_total_time = <optimized out>
#4  _dl_start (arg=0x7fffffffd9f0) at rtld.c:581
No locals.
#5  0x00007ffff7fe3dc8 in _start () from /lib64/ld-linux-x86-64.so.2
        TD_SLEEP = TD_SLEEP
        TD_CREATE = TD_CREATE
        _URC_FATAL_PHASE1_ERROR = _URC_FATAL_PHASE1_ERROR
        TD_CATCHSIG = TD_CATCHSIG
        cet_permissive = cet_permissive
        plt_rewrite_jmp = plt_rewrite_jmp
        TD_LOCK_TRY = TD_LOCK_TRY
        PREFERRED_FEATURE_INDEX_1 = PREFERRED_FEATURE_INDEX_1
        PREFERRED_FEATURE_INDEX_MAX = PREFERRED_FEATURE_INDEX_MAX
        arch_kind_unknown = arch_kind_unknown
        TD_SWITCHFROM = TD_SWITCHFROM
        cache_extension_tag_generator = cache_extension_tag_generator
        cache_extension_tag_glibc_hwcaps = cache_extension_tag_glibc_hwcaps
        _URC_INSTALL_CONTEXT = _URC_INSTALL_CONTEXT
        _bitindex_arch_Prefer_MAP_32BIT_EXEC = _bitindex_arch_Prefer_MAP_32BIT_EXEC
        TD_DEATH = TD_DEATH
        RT_CONSISTENT = RT_CONSISTENT
        LA_ACT_CONSISTENT = LA_ACT_CONSISTENT
        rtld_mode_verify = rtld_mode_verify
        TD_MAX_EVENT_NUM = TD_TIMEOUT
        RT_DELETE = RT_DELETE
        relocate_time = 0
        DL_LOOKUP_ADD_DEPENDENCY = DL_LOOKUP_ADD_DEPENDENCY
        TD_READY = TD_READY
        _bitindex_arch_Slow_SSE4_2 = _bitindex_arch_Slow_SSE4_2
        rtld_mode_help = rtld_mode_help
        cpuid_register_index_eax = cpuid_register_index_eax
        CPUID_INDEX_1 = CPUID_INDEX_1
        CPUID_INDEX_7 = CPUID_INDEX_7
        CPUID_INDEX_80000001 = CPUID_INDEX_80000001
        CPUID_INDEX_D_ECX_1 = CPUID_INDEX_D_ECX_1
        CPUID_INDEX_80000007 = CPUID_INDEX_80000007
        CPUID_INDEX_80000008 = CPUID_INDEX_80000008
        CPUID_INDEX_7_ECX_1 = CPUID_INDEX_7_ECX_1
        CPUID_INDEX_19 = CPUID_INDEX_19
        CPUID_INDEX_14_ECX_0 = CPUID_INDEX_14_ECX_0
        CPUID_INDEX_24_ECX_0 = CPUID_INDEX_24_ECX_0
        CPUID_INDEX_MAX = CPUID_INDEX_MAX
        dso_sort_algorithm_original = dso_sort_algorithm_original
        TD_CONCURRENCY = TD_CONCURRENCY
        lt_executable = lt_executable
        cpuid_register_index_ebx = cpuid_register_index_ebx
        _bitindex_arch_I686 = _bitindex_arch_I686
        cache_extension_count = cache_extension_count
        PTHREAD_MUTEX_TIMED_NP = PTHREAD_MUTEX_TIMED_NP
        PTHREAD_MUTEX_RECURSIVE_NP = PTHREAD_MUTEX_RECURSIVE_NP
        PTHREAD_MUTEX_ERRORCHECK_NP = PTHREAD_MUTEX_ERRORCHECK_NP
        PTHREAD_MUTEX_ADAPTIVE_NP = PTHREAD_MUTEX_ADAPTIVE_NP
        PTHREAD_MUTEX_NORMAL = PTHREAD_MUTEX_TIMED_NP
        PTHREAD_MUTEX_RECURSIVE = PTHREAD_MUTEX_RECURSIVE_NP
        PTHREAD_MUTEX_ERRORCHECK = PTHREAD_MUTEX_ERRORCHECK_NP
        PTHREAD_MUTEX_DEFAULT = PTHREAD_MUTEX_TIMED_NP
        PTHREAD_MUTEX_FAST_NP = PTHREAD_MUTEX_TIMED_NP
        TD_REAP = TD_REAP
        _bitindex_arch_Avoid_Non_Temporal_Memset = _bitindex_arch_Avoid_Non_Temporal_Memset
        DL_LOOKUP_RETURN_NEWEST = DL_LOOKUP_RETURN_NEWEST
        _URC_HANDLER_FOUND = _URC_HANDLER_FOUND
        cpuid_register_index_ecx = cpuid_register_index_ecx
        _bitindex_arch_Avoid_Short_Distance_REP_MOVSB = _bitindex_arch_Avoid_Short_Distance_REP_MOVSB
        _bitindex_arch_Prefer_FSRM = _bitindex_arch_Prefer_FSRM
        lc_property_unknown = lc_property_unknown
        _bitindex_arch_Fast_Unaligned_Load = _bitindex_arch_Fast_Unaligned_Load
        lt_library = lt_library
        cpuid_register_index_edx = cpuid_register_index_edx
        rtld_mode_list_diagnostics = rtld_mode_list_diagnostics
        start_time = 11764120506900
        _URC_NO_REASON = _URC_NO_REASON
        _bitindex_arch_Prefer_PMINUB_for_stringop = _bitindex_arch_Prefer_PMINUB_for_stringop
        plt_rewrite_jmpabs = plt_rewrite_jmpabs
        arch_kind_other = arch_kind_other
        rtld_mode_list = rtld_mode_list
        _dl_rtld_libname = {name = 0x200200 "/lib64/ld-linux-x86-64.so.2", next = 0x7ffff7ffe220 <newname>, dont_free = 0}
        TD_IDLE = TD_IDLE
        unknown = unknown
        _URC_FATAL_PHASE2_ERROR = _URC_FATAL_PHASE2_ERROR
        cet_elf_property = cet_elf_property
        RT_ADD = RT_ADD
        _bitindex_arch_Fast_Rep_String = _bitindex_arch_Fast_Rep_String
        _bitindex_arch_MathVec_Prefer_No_AVX512 = _bitindex_arch_MathVec_Prefer_No_AVX512
        _bitindex_arch_Fast_Copy_Backward = _bitindex_arch_Fast_Copy_Backward
        _bitindex_arch_AVX_Fast_Unaligned_Load = _bitindex_arch_AVX_Fast_Unaligned_Load
        existing = existing
        _URC_NORMAL_STOP = _URC_NORMAL_STOP
        lc_property_none = lc_property_none
        nonexisting = nonexisting
        load_time = 0
        TD_PREEMPT = TD_PREEMPT
        TD_TIMEOUT = TD_TIMEOUT
        TD_ALL_EVENTS = TD_ALL_EVENTS
        _URC_END_OF_STACK = _URC_END_OF_STACK
        _bitindex_arch_Prefer_No_AVX512 = _bitindex_arch_Prefer_No_AVX512
        arch_kind_intel = arch_kind_intel
        rtld_mode_trace = rtld_mode_trace
        rtld_mode_list_tunables = rtld_mode_list_tunables
        _bitindex_arch_Prefer_ERMS = _bitindex_arch_Prefer_ERMS
        cet_always_on = cet_always_on
        dso_sort_algorithm_dfs = dso_sort_algorithm_dfs
        LA_ACT_DELETE = LA_ACT_DELETE
        TD_SWITCHTO = TD_SWITCHTO
        cet_always_off = cet_always_off
        _bitindex_arch_Slow_BSF = _bitindex_arch_Slow_BSF
        arch_kind_zhaoxin = arch_kind_zhaoxin
        arch_kind_amd = arch_kind_amd
        _URC_FOREIGN_EXCEPTION_CAUGHT = _URC_FOREIGN_EXCEPTION_CAUGHT
        _bitindex_arch_Prefer_No_VZEROUPPER = _bitindex_arch_Prefer_No_VZEROUPPER
        TD_EVENT_NONE = TD_ALL_EVENTS
        TD_EVENTS_ENABLE = TD_EVENTS_ENABLE
        rtld_mode_normal = rtld_mode_normal
        TD_MIN_EVENT_NUM = TD_READY
        lc_property_valid = lc_property_valid
        LA_ACT_ADD = LA_ACT_ADD
        TD_PRI_INHERIT = TD_PRI_INHERIT
        _bitindex_arch_Fast_Unaligned_Copy = _bitindex_arch_Fast_Unaligned_Copy
        _URC_CONTINUE_UNWIND = _URC_CONTINUE_UNWIND
        DL_LOOKUP_FOR_RELOCATE = DL_LOOKUP_FOR_RELOCATE
        lt_loaded = lt_loaded
        plt_rewrite_none = plt_rewrite_none
        DL_LOOKUP_GSCOPE_LOCK = DL_LOOKUP_GSCOPE_LOCK
        _dl_rtld_libname2 = {name = 0x0, next = 0x0, dont_free = 0}
        _bitindex_arch_I586 = _bitindex_arch_I586
        __GI__dl_argv = 0x7fffffffd9f8
        _rtld_global_ro = {_dl_debug_mask = 0, _dl_platform = 0x7fffffffde19 "x86_64", _dl_platformlen = 6, _dl_pagesize = 4096, _dl_minsigstacksize = 1776, _dl_inhibit_cache = 0, 
          _dl_initial_searchlist = {r_list = 0x0, r_nlist = 0}, _dl_clktck = 100, _dl_verbose = 0, _dl_debug_fd = 2, _dl_lazy = 1, _dl_bind_not = 0, _dl_dynamic_weak = 0, _dl_fpu_control = 895, 
          _dl_hwcap = 2, _dl_auxv = 0x7fffffffdc98, _dl_x86_cpu_features = {basic = {kind = arch_kind_amd, max_cpuid = 16, family = 23, model = 96, stepping = 1}, features = {{{cpuid_array = {
                    8785665, 526336, 2128097803, 395049983}, cpuid = {eax = 8785665, ebx = 526336, ecx = 2128097803, edx = 395049983}}, {active_array = {0, 0, 2128097795, 394821904}, active = {
                    eax = 0, ebx = 0, ecx = 2128097795, edx = 394821904}}}, {{cpuid_array = {0, 563909033, 4194308, 0}, cpuid = {eax = 0, ebx = 563909033, ecx = 4194308, edx = 0}}, {active_array = {
                    0, 562823465, 4194304, 0}, active = {eax = 0, ebx = 562823465, ecx = 4194304, edx = 0}}}, {{cpuid_array = {8785665, 0, 1975662591, 802421759}, cpuid = {eax = 8785665, ebx = 0, 
                    ecx = 1975662591, edx = 802421759}}, {active_array = {0, 0, 353, 134217728}, active = {eax = 0, ebx = 0, ecx = 353, edx = 134217728}}}, {{cpuid_array = {15, 832, 0, 0}, cpuid = {
                    eax = 15, ebx = 832, ecx = 0, edx = 0}}, {active_array = {7, 0, 0, 0}, active = {eax = 7, ebx = 0, ecx = 0, edx = 0}}}, {{cpuid_array = {0, 27, 0, 26521}, cpuid = {eax = 0, 
                    ebx = 27, ecx = 0, edx = 26521}}, {active_array = {0, 0, 0, 0}, active = {eax = 0, ebx = 0, ecx = 0, edx = 0}}}, {{cpuid_array = {12336, 151844695, 24583, 65536}, cpuid = {
                    eax = 12336, ebx = 151844695, ecx = 24583, edx = 65536}}, {active_array = {0, 512, 0, 0}, active = {eax = 0, ebx = 512, ecx = 0, edx = 0}}}, {{cpuid_array = {0, 0, 0, 0}, cpuid = {
                    eax = 0, ebx = 0, ecx = 0, edx = 0}}, {active_array = {0, 0, 0, 0}, active = {eax = 0, ebx = 0, ecx = 0, edx = 0}}}, {{cpuid_array = {0, 0, 0, 0}, cpuid = {eax = 0, ebx = 0, 
                    ecx = 0, edx = 0}}, {active_array = {0, 0, 0, 0}, active = {eax = 0, ebx = 0, ecx = 0, edx = 0}}}, {{cpuid_array = {0, 0, 0, 0}, cpuid = {eax = 0, ebx = 0, ecx = 0, edx = 0}}, {
                  active_array = {0, 0, 0, 0}, active = {eax = 0, ebx = 0, ecx = 0, edx = 0}}}, {{cpuid_array = {0, 0, 0, 0}, cpuid = {eax = 0, ebx = 0, ecx = 0, edx = 0}}, {active_array = {0, 0, 0, 
                    0}, active = {eax = 0, ebx = 0, ecx = 0, edx = 0}}}}, preferred = {704}, isa_1 = 7, xsave_state_size = 960, xsave_state_full_size = 960, data_cache_size = 32768, 
            shared_cache_size = 4194304, non_temporal_threshold = 3145728, memset_non_temporal_threshold = 3145728, rep_movsb_threshold = 3145728, rep_movsb_stop_threshold = 3145728, 
            rep_stosb_threshold = 18446744073709551615, level1_icache_size = 32768, level1_icache_linesize = 64, level1_dcache_size = 32768, level1_dcache_assoc = 8, level1_dcache_linesize = 64, 
            level2_cache_size = 524288, level2_cache_assoc = 8, level2_cache_linesize = 64, level3_cache_size = 4194304, level3_cache_assoc = 16, level3_cache_linesize = 64, level4_cache_size = 0, 
            cachesize_non_temporal_divisor = 4}, _dl_x86_hwcap_flags = {"sse2\000\000\000\000", "x86_64\000\000", "avx512_1"}, _dl_x86_tlsdesc_dynamic = 0x7ffff7fdbb60 <_dl_tlsdesc_dynamic_xsavec>, 
          _dl_x86_64_runtime_resolve = 0x7ffff7fd9660 <_dl_runtime_resolve_xsavec>, _dl_inhibit_rpath = 0x0, _dl_origin_path = 0x0, _dl_tls_static_size = 0, _dl_tls_static_align = 0, 
          _dl_tls_static_surplus = 0, _dl_profile = 0x0, _dl_profile_output = 0x7ffff7ff3e3e "/var/tmp", _dl_init_all_dirs = 0x0, _dl_sysinfo_dso = 0x7ffff7fc5000, _dl_sysinfo_map = 0x7ffff7ffe8c0, 
          _dl_vdso_clock_gettime64 = 0x7ffff7fc5a60 <clock_gettime>, _dl_vdso_gettimeofday = 0x7ffff7fc57b0 <gettimeofday>, _dl_vdso_time = 0x7ffff7fc5a30 <time>, 
          _dl_vdso_getcpu = 0x7ffff7fc5e10 <getcpu>, _dl_vdso_clock_getres_time64 = 0x7ffff7fc5da0 <clock_getres>, _dl_hwcap2 = 2, _dl_hwcap3 = 0, _dl_hwcap4 = 0, 
          _dl_dso_sort_algo = dso_sort_algorithm_dfs, _dl_debug_printf = 0x7ffff7fd4010 <_dl_debug_printf>, _dl_mcount = 0x7ffff7fe2640 <__GI__dl_mcount>, 
          _dl_lookup_symbol_x = 0x7ffff7fd0970 <_dl_lookup_symbol_x>, _dl_open = 0x7ffff7fd2850 <_dl_open>, _dl_close = 0x7ffff7fc9640 <_dl_close>, _dl_catch_error = 0x7ffff7fc8650 <_dl_catch_error>, 
          _dl_error_free = 0x7ffff7fcaa60 <_dl_error_free>, _dl_tls_get_addr_soft = 0x7ffff7fd8e40 <_dl_tls_get_addr_soft>, _dl_libc_freeres = 0x7ffff7fe1790 <__rtld_libc_freeres>, 
          _dl_find_object = 0x7ffff7fea020 <__GI___dl_find_object>, _dl_dlfcn_hook = 0x0, _dl_audit = 0x0, _dl_naudit = 0}
        _dl_argc = 1
        __rtld_tls_init_tp_called = false
        __pointer_chk_guard_local = 0
        _rtld_global = {_dl_ns = {{_ns_loaded = 0x7ffff7ffe2e0, _ns_nloaded = 3, _ns_main_searchlist = 0x0, _ns_global_scope_alloc = 0, _ns_global_scope_pending_adds = 0, libc_map = 0x0, 
              _ns_unique_sym_table = {lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, 
                    __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, entries = 0x0, size = 0, n_elements = 0, free = 0x0}, _ns_debug = {base = {r_version = 0, 
                  r_map = 0x0, r_brk = 0, r_state = RT_CONSISTENT, r_ldbase = 0}, r_next = 0x0}}, {_ns_loaded = 0x0, _ns_nloaded = 0, _ns_main_searchlist = 0x0, _ns_global_scope_alloc = 0, 
              _ns_global_scope_pending_adds = 0, libc_map = 0x0, _ns_unique_sym_table = {lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, 
                      __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}}, entries = 0x0, size = 0, n_elements = 0, free = 0x0}, _ns_debug = {
                base = {r_version = 0, r_map = 0x0, r_brk = 0, r_state = RT_CONSISTENT, r_ldbase = 0}, r_next = 0x0}} <repeats 15 times>}, _dl_nns = 1, _dl_load_lock = {mutex = {__data = {__lock = 0, 
                __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, 
              __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, _dl_load_write_lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, 
                __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, 
          _dl_load_tls_lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, 
              __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, _dl_load_adds = 3, _dl_initfirst = 0x0, _dl_profile_map = 0x0, _dl_num_relocations = 5, 
          _dl_num_cache_relocations = 0, _dl_all_dirs = 0x7ffff7ffef30, _dl_rtld_map = {l_addr = 140737353904128, l_name = 0x200200 "/lib64/ld-linux-x86-64.so.2", l_ld = 0x7ffff7ffce80, 
            l_next = 0x7ffff7ffe8c0, l_prev = 0x7ffff7ffe2e0, l_real = 0x7ffff7ffdab0 <_rtld_global+2736>, l_ns = 0, l_libname = 0x7ffff7ffe260 <_dl_rtld_libname>, l_info = {0x0, 0x0, 0x0, 0x0, 0x0, 
              0x7ffff7ffcea0, 0x7ffff7ffceb0, 0x7ffff7ffcee0, 0x7ffff7ffcef0, 0x7ffff7ffcf00, 0x7ffff7ffcec0, 0x7ffff7ffced0, 0x0, 0x0, 0x7ffff7ffce80, 0x0 <repeats 15 times>, 0x7ffff7ffcf30, 0x0, 
              0x0, 0x0, 0x0, 0x7ffff7ffcf70, 0x7ffff7ffcf60, 0x7ffff7ffcf80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffff7ffcf20, 0x7ffff7ffcf10, 0x7ffff7ffcf40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
              0x0, 0x7ffff7ffcf50, 0x0 <repeats 25 times>, 0x7ffff7ffce90}, l_phdr = 0x7ffff7fc7040, l_entry = 0, l_phnum = 11, l_ldnum = 0, l_searchlist = {r_list = 0x0, r_nlist = 0}, 
            l_symbolic_searchlist = {r_list = 0x0, r_nlist = 0}, l_loader = 0x0, l_versions = 0x0, l_nversions = 0, l_nbuckets = 71, l_gnu_bitmask_idxbits = 3, l_gnu_shift = 8, 
            l_gnu_bitmask = 0x7ffff7fc7320, {l_gnu_buckets = 0x7ffff7fc7340, l_chain = 0x7ffff7fc7340}, {l_gnu_chain_zero = 0x7ffff7fc7458, l_buckets = 0x7ffff7fc7458}, l_direct_opencount = 0, 
            l_type = lt_library, l_dt_relr_ref = 0, l_relocated = 1, l_init_called = 0, l_global = 0, l_reserved = 0, l_main_map = 0, l_visited = 0, l_map_used = 0, l_map_done = 0, 
            l_phdr_allocated = 0, l_soname_added = 0, l_faked = 0, l_need_tls_init = 0, l_auditing = 0, l_audit_any_plt = 0, l_removed = 0, l_contiguous = 0, l_free_initfini = 0, l_ld_readonly = 0, 
            l_find_object_processed = 0, l_nodelete_active = false, l_nodelete_pending = false, l_has_jump_slot_reloc = false, l_property = lc_property_unknown, l_x86_feature_1_and = 0, 
            l_x86_isa_1_needed = 0, l_1_needed = 0, l_rpath_dirs = {dirs = 0x0, malloced = 0}, l_reloc_result = 0x0, l_versyms = 0x0, l_origin = 0x0, l_map_start = 140737353904128, 
            l_map_end = 140737354130144, l_scope_mem = {0x0, 0x0, 0x0, 0x0}, l_scope_max = 0, l_scope = 0x0, l_local_scope = {0x0, 0x0}, l_file_id = {dev = 0, ino = 0}, l_runpath_dirs = {dirs = 0x0, 
              malloced = 0}, l_initfini = 0x0, l_reldeps = 0x0, l_reldepsmax = 0, l_used = 0, l_feature_1 = 0, l_flags_1 = 0, l_flags = 0, l_idx = 0, l_mach = {plt = 0, gotplt = 0, 
              tlsdesc_table = 0x0}, l_lookup_cache = {sym = 0x0, type_class = 0, value = 0x0, ret = 0x0}, l_tls_initimage = 0x0, l_tls_initimage_size = 0, l_tls_blocksize = 0, l_tls_align = 0, 
            l_tls_firstbyte_offset = 0, l_tls_offset = 0, l_tls_modid = 0, l_tls_dtor_count = 0, l_relro_addr = 213728, l_relro_size = 7456, l_serial = 0}, _dl_rtld_auditstate = {{cookie = 0, 
              bindflags = 0} <repeats 16 times>}, _dl_x86_feature_1 = 0, _dl_x86_feature_control = {ibt = cet_elf_property, shstk = cet_elf_property, plt_rewrite = plt_rewrite_none}, 
          _dl_stack_flags = 6, _dl_tls_dtv_gaps = false, _dl_tls_max_dtv_idx = 1, _dl_tls_dtv_slotinfo_list = 0x0, _dl_tls_static_nelem = 0, _dl_tls_static_used = 0, _dl_tls_static_optional = 0, 
          _dl_initial_dtv = 0x0, _dl_tls_generation = 0, _dl_scope_free_list = 0x0, _dl_stack_used = {next = 0x7ffff7ffe0b8 <_rtld_global+4280>, prev = 0x7ffff7ffe0b8 <_rtld_global+4280>}, 
          _dl_stack_user = {next = 0x7ffff7ffe0c8 <_rtld_global+4296>, prev = 0x7ffff7ffe0c8 <_rtld_global+4296>}, _dl_stack_cache = {next = 0x7ffff7ffe0d8 <_rtld_global+4312>, 
            prev = 0x7ffff7ffe0d8 <_rtld_global+4312>}, _dl_stack_cache_actsize = 0, _dl_in_flight_stack = 0, _dl_stack_cache_lock = 0}
#6  0x0000000000000001 in ?? ()
No symbol table info available.
#7  0x00007fffffffde29 in ?? ()
No symbol table info available.
#8  0x0000000000000000 in ?? ()
No symbol table info available.

[Waqar144]

I use mold as my default linker. I switched back to bfd and the issue doesn't happen. Using lld also crashes. Maybe related? llvm/llvm-project#107749

[flysand7]

So for testing I made the following change
[...]
This might all be totally bogus though.

Yes, and I would not recommend running your executables without a dynamic linker.

For zig, this section doesn't exist.

I don't remember exactly, but I'm pretty sure there's a difference between some types of elf files where some of them will have a .interp section and others wouldn't. Executables and Shared objects? Maybe that's the difference. Also might be a PIE/PIC difference. Worth investigating. In the issue you linked there was also some discussion that -pie fixed things.

I'm 90% sure this isn't an Odin bug.

[Waqar144]

Yes, and I would not recommend running your executables without a dynamic linker.

Sure. But if the binary doesn't use any dynamic libs then at least bfd removes the dynamic linker it seems. LLD doesn't and thus things dont work.

But yeah, I agree that its not an odin bug 100% since bfd just works.

[laytan]

Can we do something to make it work with lld? Since it's a supported linker.

[flysand7]

bfd removes the dynamic linker

It removes the .interp section, your program might still be running through a dynamic linker. You may not be able to catch its execution with gdb, because by default gdb skips over the execution of the default linker. Though I actually forgot how its supposed to work down there. I'll try making a repro and see what I can find.

[flysand7]

How did you get glibc on your alpine setup?

[Waqar144]

If you are asking me: I am not using Alpine. I am using Manjaro linux

[Waqar144]

It removes the .interp section, your program might still be running through a dynamic linker.

Unless I misunderstand things, its a statically linked binary. So that doesn't make sense. See https://lwn.net/Articles/631631/

Will gdb skip execution if I use starti?

[flysand7]

starti doesn't skip execution of the dynamic linker iirc.

Also yeah I got confused, I thought you were the OP haha

[amarz45]

Sorry for responding so late, but compiling with -no-thread-local makes the program run normally. I feel like this is just a workaround though, because that means you give up multi-threading, which is not the case if you don’t pass in -no-crt.

[flysand7]

You feel like it is a workaround because, well, it is. There are a lot of problems regarding making threads work with -no-libc and still making anything coherent enough to be useable. This is where Linux and Windows differ fundamentally. And that difference is quite simple: Dynamic linkers.

On linux, when you run a program (or ask the kernel to run a program), the kernel loads the program into the memory but doesn't run it right away. Instead it scans the file for .interp section, which contains a path to the dynamic linker. Linux then runs that dynamic linker and that is the program that is responsible for initializing your stack layout, heap, thread-local storage, as well as dynamically linking (hence the name) your program to other .so files on start. After all that it runs your executable's entry point (typically libc's _start).

Here's the crux of the issue: There are different libc's and different libc's typically have different TLS layouts that are initialized by their respective dynamic linkers. Musl has its own dynamic linker, and Glibc has its own. That is kind of a contradiction: Even though you're compiling with -no-crt you still depend on its part.

So regardless, even if we manage to somehow allow threads with -no-crt, you can not link to any libraries that uses thread-locals or threads. This includes NVidia graphics drivers, as they store a lot of the state in userspace's thread locals. So now the question becomes: Is that useful, or even, what do you want to do with that?

I am sure we can make threads work with -no-crt, but only with the restriction above and provided that we basically implement our own dynamic linker (or, a part of it) inside the base library, which involves a non-trivial amount of assembly and maybe some weird pointer magic. So yeah for now it's just a workaround to prevent further issues being created by masquerading this behaviour as intentional.

If you have a reason for building a -no-crt program that uses threads, I would gladly hear about it. I'm just not seeing it myself right now.

[amarz45]

My rationale for using no-crt is that I want to build a fully static executable. By default, Odin-built executables are statically linked, with the exception of libc. In contrast, Zig builds fully static executables by default. Could this be done for Odin?

[flysand7]

Please help me understand, why do you need a statically linked executable?

[amarz45]

Mainly for portability reasons. I want to distribute binaries that will work on all Linux's.