Component-local startup code and linker scripts

All Genode components including core rely on the same startup code, which is roughly outlined at the end of Section Component creation. This section revisits the required steps in more detail and refers to the corresponding points in the source code. Furthermore, it provides background information about the linkage of components, which is closely related to the startup code.

Linker scripts

Under the hood, the Genode build system uses three different linker scripts located at _repos/base/src/ld/_:

genode.ld

is used for statically linked components, including core,

genode_dyn.ld

is used for dynamically linked components, i.e., components that are linked against at least one shared library,

genode_rel.ld

is used for shared libraries.

Additionally, there exists a special linker script for the dynamic linker (Section Dynamic linker).

Each program image generated by the linker generally consists of three parts, which appear consecutively in the component's virtual memory.

  1. A read-only "text" part contains sections for code, read-only data, and the list of global constructors and destructors.

    The startup code is placed in a dedicated section .text.crt0, which appears right at the start of the segment. Thereby the link address of the component is known to correspond to the ELF entrypoint (the first instruction of the assembly startup code). This is useful when converting the ELF image of the base-hw version of core into a raw binary. Such a raw binary can be loaded directly into the memory of the target platform without the need for an ELF loader.

    The mechanisms for generating the list of constructors and destructors differ between CPU architecture and are defined by the architecture's ABI. On x86, the lists are represented by .ctors.* and .dtors.*. On ARM, the information about global constructors is represented by .init_array and there is no visible information about global destructors.

  2. A read-writable "data" part that is pre-populated with data.

  3. A read-writable "bss" part that is not physically present in the binary but known to be zero-initialized when the ELF image is loaded.

The link address is not defined in the linker script but specified as linker argument. The default link address is specified in a platform-specific spec file, e.g., repos/base-nova/mk/spec/nova.mk for the NOVA platform. Components that need to organize their virtual address space in a special way (e.g., a virtual machine monitor that co-locates the guest-physical address space with its virtual address space) may specify link addresses that differ from the default one by overriding the LD_TEXT_ADDR value.

ELF entry point

As defined at the start of the linker script via the ENTRY directive, the ELF entrypoint is the function _start. This function is located at the very beginning of the .text.crt0 section. See the Section Startup code for more details.

Symbols defined by the linker script

The following symbols are defined by the linker script and used by the base framework.

_prog_img_beg, _prog_img_data, _prog_img_end

Those symbols mark the start of the "text" part, the start of the "data" part (the end of the "text" part), and the end of the "bss" part. They are used by core to exclude those virtual memory ranges from the core's virtual-memory allocator (core-region allocator).

_parent_cap, _parent_cap_thread_id, _parent_cap_local_name

Those symbols are located at the beginning of the "data" part. During the ELF loading of a new component, the parent writes information about the parent capability to this location (the start of the first read-writable ELF segment). See the corresponding code in the Loaded_executable constructor in base/src/lib/base/child_process.cc. The use of the information depends on the base platform. E.g., on a platform where a capability is represented by a tuple of a global thread ID and an object ID such as OKL4 and L4ka::Pistachio, the information is taken as verbatim values. On platforms that fully support capability-based security without the use of any form of a global name to represent a capability, the information remains unused. Here, the parent capability is represented by the same known local name in all components.

Even though the linker scripts are used across all base platforms, they contain a few platform-specific supplements that are needed to support the respective kernel ABIs. For example, the definition of the symbol __l4sys_invoke_indirect is needed only on the Fiasco.OC platform and is unused on the other base platforms. Please refer to the comments in the linker script for further explanations.

Startup code

The execution of the initial thread of a new component starts at the ELF entry point, which corresponds to the _start function. This is an assembly function defined in repos/base/src/lib/startup/spec/<arch>/crt0.s where <arch> is the CPU architecture (x86_32, x86_64, or ARM).

Assembly startup code

The assembly startup code is position-independent code (PIC). Because the Genode base libraries are linked against both statically-linked and dynamically linked executables, they have to be compiled as PIC code. To be consistent with the base libraries, the startup code needs to be position-independent, too.

The code performs the following steps:

  1. Saving the initial state of certain CPU registers. Depending on the used kernel, these registers carry information from the kernel to the core component. More details about this information are provided by Section Bootstrapping and allocator setup. The initial register values are saved in global variables named _initial_<register>. The global variables are located in the BSS segment. Note that those variables are used solely by core.

  2. Setting up the initial stack. Before the assembly code can call any higher-level C function, the stack pointer must be initialized to point to the top of a valid stack. The initial stack is located in the BSS section and referred to by the symbol _stack_high. However, having a stack located within the BSS section is dangerous. If it overflows (e.g., by declaring large local variables, or by recursive function calls), the stack would silently overwrite parts of the BSS and DATA sections located below the lower stack boundary. For prior known code, the stack can be dimensioned to a reasonable size. But for arbitrary application code, no assumption about the stack usage can be made. For this reason, the initial stack cannot be used for the entire lifetime of the component. Before any component-specific code is called, the stack needs to be relocated to another area of the virtual address space where the lower bound of the stack is guarded by empty pages. When using such a "real" stack, a stack overflow will produce a page fault, which can be handled or at least immediately detected. The initial stack is solely used to perform the steps required to set up the real stack. Because those steps are the same for all components, the usage of the initial stack is bounded.

  3. Because the startup code is used by statically linked components as well as the dynamic linker, the startup immediately calls the init_rtld hook function. For regular components, the function does not do anything. The default implementation in init_main_thread.cc at repos/base/src/lib/startup/ is a weak function. The dynamic linker provides a non-weak implementation, which allows the linker to perform initial relocations of itself very early at the dynamic linker's startup.

  4. By calling the init_main_thread function defined in repos/base/src/lib/startup/init_main_thread.cc, the assembly code triggers the execution of all the steps needed for the creation of the real stack. The function is implemented in C++, uses the initial stack, and returns the address of the real stack.

  5. With the new stack pointer returned by init_main_thread, the assembly startup code is able to switch the stack pointer from the initial stack to the real stack. From this point on, stack overflows cannot easily corrupt any data.

  6. With the real stack in place, the assembly code finally passes the control over to the C++ startup code provided by the _main function.

Initialization of the real stack along with the Genode environment

As mentioned above, the assembly code calls the init_main_thread function (located in repos/base/src/lib/startup/init_main_thread.cc) for setting up the real stack for the program. For placing a stack in a dedicated portion of the component's virtual address space, the function needs to overcome two principle problems:

In order to solve both problems, the function needs to obtain the capability for its PD session from its parent. This comes down to the need to perform RPC calls. First, for requesting the PD session capability from the parent, and second, for invoking the session capability to perform the RAM allocation and region-map attach operations.

The RPC mechanism is based on C++. In particular, the mechanism supports the propagation of C++ exceptions across RPC interfaces. Hence, before being able to perform RPC calls, the program must initialize the C++ runtime including the exception-handling support. The initialization of the C++ runtime, in turn, requires support for dynamically allocating memory. Hence, a heap must be available. This chain of dependencies ultimately results in the need to construct the entire Genode environment as a side effect of initializing the real stack of the program.

During the construction of the Genode environment, the program requests its own CPU, PD, and LOG sessions from its parent.

With the environment constructed, the program is able to interact with its own PD session and can principally realize the initialization of the real stack. However, instead of merely allocating a new RAM dataspace and attaching the dataspace to the address space of the PD session, a so-called stack area is used. The stack area is a secondary region map that is attached as a dataspace to the component's address-space region map. This way, virtual-memory allocations within the stack area can be managed manually. I.e., the spaces between the stacks of different threads are guaranteed to remain free from any attached dataspaces. The stack area of a component is created as part of the component's PD session. The environment initialization code requests its region-map capability via Pd_session::stack_area and attaches it as a managed dataspace to the component's address space.

Component-dependent startup code

With the Genode environment constructed and the initial stack switched to a proper stack located in the stack area, the component-dependent startup code of the _main function in repos/base/src/lib/startup/_main.cc can be executed. This code is responsible for calling the global constructors of the program before calling the program's main function.

In accordance to the established signature of the main function, taking an argument list and an environment as arguments, the startup code supplies these arguments but uses dummy default values. However, since the values are taken from the global variables genode_argv, genode_argc, and genode_envp, a global constructor is able to override the default values.

The startup code in _main.cc is accompanied with support for atexit handling. The atexit mechanism allows for the registration of handlers to be called at the exit of the program. It is provided in the form of a POSIX API by the C runtime. But it is also used by the compiler to schedule the execution of the destructors of function-local static objects. For the latter reason, the atexit mechanism cannot be merely provided by the (optional) C runtime but must be supported by the base library.