Genode Porting Guide: Porting a program to natively run on Genode
As an example on how to create a native port of a program for Genode, we will describe the porting of DosBox more closely. Hereby, each of the steps outlined in the previous section will be discussed in detail.
Check requirements/dependencies
In the first step, we build DosBox for Linux/x86 to obtain needed information. Nowadays, most applications use a build-tool like Autotools or something similar that will generate certain files (e.g., config.h). These files are needed to successfully compile the program. Naturally they are required on Genode as well. Since Genode does not use the original build tool of the program for native ports, it is appropriate to copy those generated files and adjust them later on to match Genode's settings.
We start by checking out the source code of DosBox from its subversion repository:
$ svn export http://svn.code.sf.net/p/dosbox/code-0/dosbox/trunk@3837 dosbox-svn-3837 $ cd dosbox-svn-3837
At this point, it is helpful to disable certain options that are not available or used on Genode just to keep the noise down:
$ ./configure --disable-opengl $ make > build.log 2>&1
After the DosBox binary is successfully built, we have a log file (build.log) of the whole build process at our disposal. This log file will be helpful later on when the target.mk file needs to be created. In addition, we will inspect the DosBox binary:
$ readelf -d -t src/dosbox|grep NEEDED 0x0000000000000001 (NEEDED) Shared library: [libasound.so.2] 0x0000000000000001 (NEEDED) Shared library: [libdl.so.2] 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libSDL-1.2.so.0] 0x0000000000000001 (NEEDED) Shared library: [libpng12.so.0] 0x0000000000000001 (NEEDED) Shared library: [libz.so.1] 0x0000000000000001 (NEEDED) Shared library: [libSDL_net-1.2.so.0] 0x0000000000000001 (NEEDED) Shared library: [libX11.so.6] 0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6] 0x0000000000000001 (NEEDED) Shared library: [libm.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Using readelf on the binary shows all direct dependencies. We now know that at least libSDL, libSDL_net, libstdc++, libpng, libz, and libm are required by DosBox. The remaining libraries are mostly mandatory on Linux and do not matter on Genode. Luckily all of these libraries are already available on Genode. For now all we have to do is to keep them in mind.
Creating the port file
Since DosBox is an application, which depends on several ported libraries (e.g., libSDL), the ports repository within the Genode source tree is a natural fit. On that account, the port file ports/ports/dosbox.port is created.
For DosBox the dosbox.port looks as follows:
LICENSE := GPLv2 VERSION := svn DOWNLOADS := dosbox.svn URL(dosbox) := http://svn.code.sf.net/p/dosbox/code-0/dosbox/trunk DIR(dosbox) := src/app/dosbox REV(dosbox) := 3837
First, we define the license, the version and the type of the source code origin. In case of DosBox, we checkout the source code from a Subversion repository. This is denoted by the .svn suffix of the item specified in the DOWNLOADS declaration. Other valid types are file (a plain file), archive (an archive of the types tar.gz, tar.xz, tgz, tar.bz2, or zip) or git (a Git repository). To checkout the source code out from the Subversion repository, we also need its URL, the revision we want to check out and the destination directory that will contain the sources afterwards. These declarations are mandatory and must always be specified. Otherwise the preparation of the port will fail.
PATCHES := $(addprefix src/app/dosbox/patches/,\
$(notdir $(wildcard $(REP_DIR)/src/app/dosbox/patches/*.patch)))
PATCH_OPT := -p2 -d src/app/dosbox
As next step, we declare all patches that are needed for the DosBox port. Since in this case, the patches are using a different path format, we have to override the default patch settings by defining the PATCH_OPT variable.
Each port file comes along with a hash file. This hash is generated by taking several sources into account. For one, the port file, each patch and the port preparation tool (tool/ports/prepare_port) are the ingredients for the hash value. If any of these files is changed, a new hash will be generated, For now, we just write "dummy" in the '_ports/ports/dosbox.hash_ file.
The DosBox port can now be prepared by executing
$ <genode-dir>/tool/ports/prepare_port dosbox
However, we get the following error message:
Error: <rep-dir>/ports/dosbox.port is out of date, expected <fingerprint>
We get this message because we had specified the "dummy" hash value in the dosbox.hash file. The prepare_port tool computes a fingerprint of the actual version of the port and compares this fingerprint with the hash value specified in dosbox.hash. The computed fingerprint can be found at <genode-dir>/contrib/dosbox-dummy/dosbox.hash. In the final step of the port, we will replace the dummy fingerprint with the actual fingerprint of the port. But before finalizing the porting work, it is practical to keep using the dummy hash and suppress the fingerprint check. This can be done by adding CHECK_HASH=no as argument to the prepare_port tool:
$ <genode-dir>/tool/ports/prepare-port dosbox CHECK_HASH=no
Check platform-dependent code
At this point, it is important to spot platform-dependent source files or rather certain functions that are not yet available on Genode. These source files should be omitted. Of course they may be used as a guidance when implementing the functionality for Genode later on, when creating the target.mk file. In particular the various cdrom_ioctl_*.cpp files are such candidates in this example.
Creating the build Makefile
Now it is time to write the build rules into the target.mk, which will be placed in ports/src/app/dosbox.
Armed with the build.log that we created while building DosBox on Linux, we assemble a list of needed source files. If an application just uses a simple Makefile and not a build tool, it might be easier to just reuse the contents of this Makefile instead.
First of all, we create a shortcut for the source directory of DosBox by calling the select_from_ports function:
DOSBOX_DIR := $(call select_from_ports,dosbox)/src/app/dosbox
Under the hood, the select_from_ports function looks up the fingerprint of the specified port by reading the corresponding
<port-name>.hash file. It then uses this hash value to construct the directory path within the <genode-dir>contrib/ directory that belongs to the matching version of the port. If there is no hash file that matches the port name, or if the port directory does not exist, the build system will back out with an error message.
Examining the log file leaves us with the following list of source files:
SRC_CC_cpu = $(notdir $(wildcard $(DOSBOX_DIR)/src/cpu/*.cpp))
SRC_CC_debug = $(notdir $(wildcard $(DOSBOX_DIR)/src/debug/*.cpp))
FILTER_OUT_dos = cdrom_aspi_win32.cpp cdrom_ioctl_linux.cpp cdrom_ioctl_os2.cpp \
cdrom_ioctl_win32.cpp
SRC_CC_dos = $(filter-out $(FILTER_OUT_dos), \
$(notdir $(wildcard $(DOSBOX_DIR)/src/dos/*.cpp)))
[…]
SRC_CC = $(notdir $(DOSBOX_DIR)/src/dosbox.cpp)
SRC_CC += $(SRC_CC_cpu) $(SRC_CC_debug) $(SRC_CC_dos) $(SRC_CC_fpu) \
$(SRC_CC_gui) $(SRC_CC_hw) $(SRC_CC_hw_ser) $(SRC_CC_ints) \
$(SRC_CC_ints) $(SRC_CC_misc) $(SRC_CC_shell)
vpath %.cpp $(DOSBOX_DIR)/src
vpath %.cpp $(DOSBOX_DIR)/src/cpu
[…]
The only variable here that is actually evaluated by Genode's build-system is SRC_CC. The rest of the variables are little helpers that make our life more comfortable.
In this case, it is mandatory to use GNUMake's notdir file name function because otherwise the compiled object files would be stored within the contrib directories. Genode runs on multiple platforms with varying architectures and mixing object files is considered harmful, which can happen easily if the application is build from the original source directory. That's why you have to use a build directory for each platform. The Genode build system will create the needed directory hierarchy within the build directory automatically. By combining GNUMake's notdir and wildcard function, we can assemble a list of all needed source files without much effort. We then use vpath to point GNUMake to the right source file within the dosbox directory.
The remaining thing to do now is setting the right include directories and proper compiler flags:
INC_DIR += $(PRG_DIR)
INC_DIR += $(DOSBOX_DIR)/include
INC_DIR += $(addprefix $(DOSBOX_DIR)/src, cpu debug dos fpu gui hardware \
hardware/serialport ints misc shell)
PRG_DIR is a special variable of Genode's build-system and its value is always the absolute path to the directory containing the target.mk file.
We copy the config.h file, which was generated in the first step to this directory and change certain parts of it to better match Genode's environment. Below is a skimmed diff of these changes:
--- config.h.orig 2013-10-21 15:27:45.185719517 +0200 +++ config.h 2013-10-21 15:36:48.525727975 +0200 @@ -25,7 +25,8 @@ /* #undef AC_APPLE_UNIVERSAL_BUILD */ /* Compiling on BSD */ -/* #undef BSD */ +/* Genode's libc is based on FreeBSD 8.2 */ +#define BSD 1 […] /* The type of cpu this target has */ -#define C_TARGETCPU X86_64 +/* we define it ourself */ +/* #undef C_TARGETCPU */ […]
Thereafter, we specify the compiler flags:
CC_OPT = -DHAVE_CONFIG_H -D_GNU_SOURCE=1 -D_REENTRANT
ifeq ($(filter-out $(SPECS),x86_32),)
INC_DIR += $(PRG_DIR)/x86_32
CC_OPT += -DC_TARGETCPU=X86
else ifeq ($(filter-out $(SPECS),x86_64),)
INC_DIR += $(PRG_DIR)/x86_64
CC_OPT += -DC_TARGETCPU=X86_64
endif
CC_WARN = -Wall
#CC_WARN += -Wno-unused-variable -Wno-unused-function -Wno-switch \
-Wunused-value -Wno-unused-but-set-variable
As noted in the commentary seen in the diff we define C_TARGETCPU and adjust the include directories ourselves according to the target architecture.
While debugging, compiler warnings for 3rd-party code are really helpful but tend to be annoying after the porting work is finished, we can remove the hashmark to keep the compiler from complaining too much.
Lastly, we need to add the required libraries, which we acquired in step 1:
LIBS += libc libm libpng sdl stdcxx zlib LIBS += libc_lwip_nic_dhcp config_args
In addition to the required libraries, a few Genode specific libraries are also needed. These libraries implement certain functions in the libc via the libc's plugin mechanism. libc_lwip_nic_dhcp, for example, is used to connect the BSD socket interface to a NIC service such as a network device driver.
Creating the run script
To ease compiling, running and debugging DosBox, we create a run script at ports/run/dosbox.run.
First, we specify the components that need to be built
set build_components {
core init drivers/audio drivers/framebuffer drivers/input
drivers/pci drivers/timer app/dosbox
}
build $build_components
and instruct tool/run to create the boot directory that hosts all binaries and files which belong to the DosBox scenario.
As the name build_components suggests, you only have to declare the components of Genode, which are needed in this scenario. All dependencies of DosBox (e.g. libSDL) will be built before DosBox itself.
Nextm we provide the scenario's configuration config:
append config {
<config>
<parent-provides>
<service name="ROM"/>
<service name="RAM"/>
<service name="IRQ"/>
<service name="IO_MEM"/>
<service name="IO_PORT"/>
<service name="CAP"/>
<service name="PD"/>
<service name="RM"/>
<service name="CPU"/>
<service name="LOG"/>
<service name="SIGNAL"/>
</parent-provides>
<default-route>
<any-service> <parent/> <any-child/> </any-service>
</default-route>
<start name="audio_drv">
<resource name="RAM" quantum="6M"/>}
<provides><service name="Audio_out"/></provides>
</start>
<start name="fb_drv">
<resource name="RAM" quantum="4M"/>
<provides><service name="Framebuffer"/></provides>
</start>
<start name="ps2_drv">
<resource name="RAM" quantum="1M"/>
<provides><service name="Input"/></provides>
</start>
<start name="timer">
<resource name="RAM" quantum="1M"/>
<provides><service name="Timer"/></provides>
</start>
<start name="dosbox">
<resource name="RAM" quantum="128M"/>
<config>
<sdl_audio_volume value="100"/>
<libc stdout="/dev/log" stderr="/dev/log">
<vfs>
<tar name="dosbox.tar"/>
<dir name="dev"> <log/> </dir>
</vfs>
</libc>
</config>
</start>
</config>}
install_config $config
The config file will be used by the init program to start all components and application of the scenario, including DosBox.
Thereafter we declare all boot modules:
set boot_modules {
core init timer audio_drv fb_drv ps2_drv ld.lib.so
libc.lib.so libm.lib.so
lwip.lib.so libpng.lib.so stdcxx.lib.so sdl.lib.so
pthread.lib.so zlib.lib.so dosbox dosbox.tar
}
build_boot_image $boot_modules
The boot modules comprise all binaries and other files like the tar archive that contains DosBox' configuration file dosbox.conf that are needed for this scenario to run sucessfully.
Finally, we set certain options, which are used when Genode is executed in Qemu and instruct tool/run to keep the scenario executing as long as it is not manually stopped:
append qemu_args " -m 256 -soundhw ac97 " run_genode_until forever
It is reasonable to write the run script in a way that makes it possible to use it for multiple Genode platforms. Debugging is often done on Genode/Linux or on another Genode platform running in Qemu but testing is normally done using actual hardware.
Compiling the program
To compile DosBox and all libraries it depends on, we execute
$ make app/dosbox
from within Genode's build directory.
We could also use the run script that we created in the previous step but that would build all components that are needed to actually run DosBox and at this point our goal is just to get DosBox compiled.
At the first attempt, the compilation stopped because g++ could not find the header file sys/timeb.h:
/src/genode/ports/contrib/dosbox-svn-3837/src/ints/bios.cpp:35:23: fatal error:
sys/timeb.h: No such file or directory
This header is part of the libc but until now there was no program, which actually used this header. So nobody noticed that it was missing. This can happen all time when porting a new application to Genode because most functionality is enabled or rather added on demand. Someone who is porting applications to Genode has to be aware of the fact that it might be necessary to extend Genode functionality by enabling so far disabled bits or implementing certain functionality needed by the application that is ported.
Since ftime(3) is a deprecated function anyway we change the code of DosBox to use gettimeofday(2).
After this was fixed, we face another problem:
/src/genode/ports/contrib/dosbox-svn-3837/src/ints/int10_vesa.cpp:48:33: error:
unable to find string literal operator ‘operator"" VERSION’
The fix is quite simple and the compile error was due to the fact that Genode uses C++11 by now. It often happens that 3rd party code is not well tested with a C++11 enabled compiler. In any case, a patch file should be created which will be applied when preparing the port.
Furthermore it would be reasonable to report the bug to the DosBox developers so it can be fixed upstream. We can then get rid of our local patch.
The next show stoppers are missing symbols in Genode's SDL library port. As it turns out, we never actually compiled and linked in the cdrom dummy code which is provided by SDL.
Running the application
DosBox was compiled successfully. Now it is time to execute the binary on Genode. Hence we use the run script we created in step 5:
$ make run/dosbox
This may take some time because all other components of the Genode OS Framework that are needed for this scenario have to be built.
Debugging the application
DosBox was successfully compiled but unfortunately it did not run. To be honest that was expected and here the fun begins.
At this point, there are several options to chose from. By running Genode/Fiasco.OC within Qemu, we can use the kernel debugger (JDB) to take a deeper look at what went wrong (e.g., backtraces of the running processes, memory dumps of the faulted DosBox process etc.). Doing this can be quite taxing but fortunately Genode runs on multiple kernels and often problems on one kernel can be reproduced on another kernel. For this reason, we choose Genode/Linux where we can use all the normal debugging tools like gdb(1), valgrind(1) and so on. Luckily for us, DosBox also fails to run on Genode/Linux. The debugging steps are naturally dependent on the ported software. In the case of DosBox, the remaining stumbling blocks were a few places where DosBox assumed Linux as a host platform.
For the sake of completeness here is a list of all files that were created by porting DosBox to Genode:
ports/ports/dosbox.hash ports/ports/dosbox.port ports/run/dosbox.run ports/src/app/dosbox/config.h ports/src/app/dosbox/patches/bios.patch ports/src/app/dosbox/patches/int10_vesa.patch ports/src/app/dosbox/target.mk ports/src/app/dosbox/x86_32/size_defs.h ports/src/app/dosbox/x86_64/size_defs.h
|
|
DosBox ported to Genode
|
Finally, after having tested that both the preparation-step and the build of DosBox work as expected, it is time to finalize the fingerprint stored in the <genode-dir>/ports/ports/dosbox.hash file. This can be done by copying the content of the <genode-dir>/contrib/dosbox-dummy/dosbox.hash file. Alternatively, you may invoke the tool/ports/update_hash tool with the port name "dosbox" as argument. The next time, you invoke the prepare_port tool, do not specify the CHECK_HASH=no argument. So the fingerprint check will validate that the dosbox.hash file corresponds to your dosbox.port file. From now on, the <genode-dir>/contrib/dosbox-dummy directory will no longer be used because the dosbox.hash file points to the port directory named after the real fingerprint.