Release notes for the Genode OS Framework 13.11
Unlike most Genode releases, which address two or three major topics, version 13.11 brings plentiful new experimental features and improvements of details. The experimental ground covered by the new release reaches from the use of the Linux TCP/IP stack as user-level library, over the native use of Qt5 QML on Genode, new FUSE-based file systems, to the exploration of C++11 for the framework. In line with the previous releases, the new version extends the coverage of device drivers, particularly for Exynos-5 SoCs and the Raspberry Pi.
Genode keeps steadily growing. With the code base approaching the number of 150 components and as many test cases, we are concerned to keep the code orthogonal and void of bugs. Of course, automated testing helps a lot. But even more importantly, we are constantly exploring possible ways to refine the Genode API such that users of the framework are saved from writing boiler-plate code, falling into C++ pitfalls, or solving complicated problems again and again. One of those problems is lock-based synchronization, which seems to be omni-present in low-level systems code, particularly on classical L4-based systems. However, taking our existing components as use cases, we came to the realization that for most of our system services, in particular device drivers, multi-threading is more of a burden than a benefit. The current framework API, however, imposes the need to use multiple threads on many components that do not benefit from parallelism. This raises the question of how we can evolve the API to become a better fit for typical use cases and relieve the component developer from dealing with complicated synchronization problems. We seek the answer to such questions through experimentation. The new server API described in Section New server API and the use of this new facility by components like the nitpicker GUI server is such an experiment.
For our mission to use Genode as a general-purpose OS, robust protocol stacks such as file systems or networking stacks are fundamental. So we are exploring different ways to bring existing implementations to Genode. For this release, we took a closer look at FUSE as a provider for file systems. Section File systems based on FUSE gives a brief overview of the results. For networking, we undertook the challenge of turning the Linux TCP/IP stack into a user-level library that can be used by Genode programs. Section Gigabit networking using the Linux TCP/IP stack explains how this TCP/IP stack can be used as an alternative to lwIP to improve gigabit-networking performance.
The recently added support for Qt5-based applications raised repeated questions about using QML, which is the most distinctive feature of Qt5 compared to Qt4. We are happy to report to have a positive answer to this question with the inclusion of initial QML support. Even though the support for QML is not mature yet, it is already possible to run the usual QML examples on Genode. Section Qt5 with support for OpenGL and QML outlines the steps for running simple QML applications.
With regard to the support for hardware platforms, version 13.11 comes with extended driver support for Exynos-5 covering HDMI and USB 3.0 mass storage, as well as all drivers needed to execute interactive Genode scenarios on the Raspberry Pi. Furthermore, we enabled the use of ARM TrustZone on i.MX53 via our custom base-hw kernel platform.
Base framework
Dynamic resource balancing
We have addressed the general problem of reassigning memory resources between subsystems in a dynamic fashion. The Genode architecture devises the explicit assignment of resources by a parent process to its children. Once assigned, the amount of memory used to remain at the disposal of the respective child subsystem until the subsystem gets destroyed by the parent.
Because Genode passes resources among processes along the branches of the process tree, the parent interface is the natural place for providing a protocol for dynamically upgrading and withdrawing resources between a parent process and its children. The protocol consists of the following functions that were added to the parent interface.
void resource_avail_sigh(Signal_context_capability)
By calling this function, the child can install a custom signal handler that gets notified on the arrival of additional resources. Such a signal gets fired by the parent after upgrading the resource quota of the child.
void resource_request(Resource_args const &)
Using this function, the child is able to apply for additional resources at its parent. The request takes the amount of desired resources as argument. A child would invoke this function if it detects scarceness of resources. If a custom signal handler is installed via resource_avail_sigh, the function returns immediately and the child must wait for the reception of the corresponding signal. If no signal handler was explicitly installed, the Genode::env() handles the request via a built-in signal handler that blocks until the parent satisfies the request.
void yield_sigh(Signal_context_capability)
By calling this function, a child installs a signal handler that gets notified about so-called yield requests from the parent. A parent issues a yield request to a subsystem to express its wish for regaining resources. It is up to the child to comply with a yield request or not. Some kinds of subsystems have meaningful ways to handle yield requests. For example, an in-memory block cache could write back the cached information and release the RAM consumed by the cache.
Resource_args yield_request()
The yield_request function returns the amount of resources desired by the parent. This information can be used by the child to tailor the resource deallocation strategy according to the needs of the parent.
void yield_response()
Once the child has taken steps to yield resources, it calls the yield_response function to inform the parent about the availability of released resources. The parent may respond to this function by withdrawing those resources from the child's RAM quota.
The protocol described above has been implemented into the base system (i.e., Genode::Parent, Genode::Child, Genode::env()) as well as the interactive console called cli_monitor. For the latter, we introduced new commands for dynamically balancing RAM between subsystems. The status command prints a table with the RAM status of each subsystem. The ram command changes the quota or a quota limit of a given subsystem. The quota limit can be defined to allow the on-demand expansion of the quota. Finally, the yield command can be used to instruct a subsystem to yield a specified amount of resources.
C++11 enabled by default
As more and more individual Genode components started using C++11 features, it is time to enable the most recent C++ standard by default. This step clears the way for using C++11 in the base API of the framework. There are many ways we may leverage the new language features to improve our code, for example variadic templates may help us to overcome current restrictions in the RPC framework regarding the number of supported RPC function arguments.
The C++11 standard is enabled by default for software built via the Genode build system. If you hit incompatibilities with existing software that relies on a prior version of the C++ standard - as is the case for some parts of Qt4 - you can override the default by manually defining CC_CXX_OPT_STD in the build-description file of the offending target.
Improved event tracing
In the previous release, we introduced Genode's event tracing facility that enables us to capture inter-process communication and component behavior with negligible overhead. To make the mechanism usable in practice, an easy-to-use front end is desired. With the current release, we have taken the first steps in this direction. The front-end API is located at base/include/base/trace/buffer.h. It hides the technical details of how a trace buffer is realized behind a convenient interface for iterating over events. Furthermore, we have added a mechanism for detecting buffer wraps and expose this information through the API.
Thanks to this convenience API, the creation of custom trace monitors becomes much more straight-forward. A simple trace-monitor implementation using the new API can be found at os/src/test/trace/.
New support for bit sets in MMIO API
Genode's API for accessing MMIO registers helps a lot to simplify the access to individual bits or consecutive bit fields of hardware registers. However, we still stumbled over use cases that the current API does not cover, particularly the access to bit fields that are not consecutive or even distributed over multiple registers. For example, values of the HDMI configuration of the Exynos-5 SoC are scattered over multiple hardware registers. Hence, we extended the MMIO API to hide those peculiarities behind an easy-to-use interface.
The idea is best illustrated by the following example of a hypothetical timer device. The bits of the clock count value are scattered across two hardware registers, the lower 6 bits of the 16-bit-wide register 0x2, and two portions of the 32-bit-wide register 0x4. A declaration of those registers would look like this:
struct Clock_2 : Register<0x2, 16>
{
struct Value : Bitfield<0, 6> { };
};
struct Clock_1 : Register<0x4, 32>
{
struct Value_2 : Bitfield<2, 13> { };
struct Value_1 : Bitfield<18, 7> { };
};
Writing a clock value needs consecutive write accesses to both registers with bit shift operations applied to the value:
write<Clock_1::Value_1>(clk); write<Clock_1::Value_2>(clk >> 7); write<Clock_2::Value>(clk >> 20);
The new Bitset_2 and Bitset_3 class templates contained in util/register.h allow the user to compose a logical bit field from 2 or 3 physical bit fields. The order of the template arguments expresses the order of physical bits in the logical bit set. Each argument can be a register, a bit field, or another bit set. The declaration of such a composed bit set for the example above looks as follows:
struct Clock : Bitset_3<Clock_1::Value_1,
Clock_1::Value_2,
Clock_2::Value> { };
With this declaration in place, the driver code becomes as simple as:
write<Clock>(clk);
Under the hood, the MMIO framework performs all needed consecutive write operations on the registers 0x2 and 0x4.
New utility for handling character buffers
Throughout Genode, we find manually instantiated char arrays and checks for null-termination scattered throughout the code base, mostly in connection with RPC functions. To avoid repetitions of such C-fashioned and bug-prone code, we added a new String template class to util/string.h. It plainly holds a null-terminated string to be stored as a member variable (e.g., a session label) or passed as RPC argument. The string class is not intended to become a full-fledged string-handling API though.
Low-level OS infrastructure
Gigabit networking using the Linux TCP/IP stack
On Genode, we used to rely on lwIP as TCP/IP stack. Network-using applications would simply link against the lwIP stack and our libc_lwip_nic_dhcp plugin to access a NIC session, as provided by a NIC driver. For using multiple networking application on the same machine, there is a NIC bridge component, which multiplexes one physical NIC to multiple virtual NICs.
While experimenting with gigabit networking, we discovered that lwIP lags far behind the performance of other modern TCP/IP stacks such as the one found in the Linux kernel. This observation prompted us to create an alternative to lwIP, which uses the TCP/IP stack of the Linux kernel. The result is called LXIP and can be found in the dde_linux repository. LXIP consists of two parts, a port of the Linux TCP/IP stack to Genode and glue code to Genode's C runtime. It comes in the form of a shared libraries named lxip.lib.so and libc_lxip.lib.so. The IP stack can be interfaced using Genode's version of libc by linking your application to the libc_lxip plugin in your target.mk file.
From the application developer's point of view, the use of LXIP instead of LwIP is as simple as replacing the libc_lwip_dhcp library with the libc_lxip library. Note however, that LXIP has larger memory demands compared to lwIP. So quota adjustments may be needed. For a quick test drive of LXIP, we have prepared a set of run scripts at libports/run/ that execute netperf using LXIP in different settings. The netperf_lxip.run script connects netperf directly with a NIC driver, the netperf_lxip_bridge.run script adds a NIC bridge as an indirection between the netperf application and the driver, and the netperf_lxip_usb30.run script targets a Gigabit network-over-USB3 network adapter.
Configuration and session-label utilities
The often-used convenience utility for accessing a process configuration used to come in the form of the header file os/config.h. But this causes aliasing problems if multiple compilation units access the config while the configuration gets dynamically updated. Moving the implementation of the accessor to the singleton object into a library solves those problems. Because of this change, all programs that rely on the Genode::config() utility need to have the config library specified in the LIBS declaration of their target.mk file.
Closely related to the process configuration, the utilities for handling session labels have slightly changed. By splitting the former Session_policy into two classes, we make it more flexible. Originally, the constructor solely accepted an args string, which made it unusable for situations where we already had extracted the session label (e.g., stored in the session meta data of a server). Now, the extraction of the label from the args string is performed by the new Session_label class instead, which, in turn, can be passed to the constructor of Session_policy. This change causes a minor API change. The following code
Session_policy policy(session_args);
must be turned into
Session_label label(session_args); Session_policy policy(label);
Serialization of signal and RPC dispatching
In Genode, inter-process communication can be implemented synchronously via RPC calls or asynchronously via signals. Most services use a combination of both mechanisms. For example, a block driver provides an RPC interface for querying the size of the block device. However, the actual transactions are communicated via shared memory and asynchronous notifications to maximize throughput. For this reason, most servers need to handle both, incoming RPCs and signals.
RPCs are dispatched by special threads called RPC entrypoints whereas signals can be waited-for by any thread, most commonly the main thread. Consequently, RPC functions and signal dispatchers are executed in the contexts of different threads. So access to the state shared by both signal handlers and RPC functions must be synchronized. This synchronization can be performed by using locks. However, lock-based synchronization is complicated and hard to test. For this reason, we introduce an alternative mechanism to serialize signal handlers with RPC functions by the means of the so-called Signal_rpc_dispatcher utility in os/signal_rpc_dispatcher.h. A signal RPC dispatcher acts as a proxy, which delegates the handling of signals to the context of an RPC entrypoint. This way, signals received by the main thread result in a process-local RPC call to an RPC entrypoint, which is naturally serialized with ordinary RPC calls. This way, both signals and RPCs are effectively handled by the same thread, which alleviates the need for synchronization.
New server API
The new signal RPC dispatcher described above offers a glimpse into the direction we want to see the Genode API evolve. Instead of handling synchronous and asynchronous communication via different mechanisms, we would like to streamline both into one solution. Right now, we have RPC entrypoints for handling RPC requests and signal receivers for handling signals. In the future, we would like to have just entrypoints, which can be associated with both RPC objects and signal dispatchers.
Because we cannot change the Genode API over night, we take gradual steps. One step is the new server library with the interface located at os/server.h. This library contains a skeleton for a common single-threaded server that wants to respond to both incoming RPC requests as well as signals. The interface provides a single Entrypoint. In contrast to an Rpc_entrypoint, a Server::Entrypoint can be associated to signal dispatchers.
Over the course of the next year, we will evaluate this idea by migrating existing servers to the new API and refining the server library as needed. If the new approach stands the test of time, we will possibly replace parts of the existing Genode API with the much simpler Server::Entrypoint interface.
Nitpicker GUI server
The nitpicker GUI server is the first server adapted to the new server API described above. Besides this change under the hood, nitpicker gained the following functional improvements:
First and foremost, the use of the server API cleared the way to let nitpicker respond to signals, particularly configuration changes at runtime. The new version is able to immediately respond to such changes including the definition of global keys, session colors, and the background color.
Global keys for the X-ray and kill modes are no longer hard-coded. The keys for toggling those functions can be defined in the nitpicker configuration as follows:
<config>
<global-keys>
<key name="KEY_SCROLLLOCK" operation="xray" />
<key name="KEY_PRINT" operation="kill" />
</global-keys>
</config>
The <global-keys> node contains the policy for handling global keys. Each <key> sub node expresses a rule for a named key. The operation attribute refers to nitpicker's built-in operations. In the example above, the X-ray mode can be activated via the scroll-lock key and the kill mode can be activated via the print key.
Alternatively to specifying an operation attribute, a key node can contain a label attribute. If specified, all events regarding the key will be reported to the client with the specified label. This enables clients to handle global shortcuts. The client with the matching label will receive all events until the number of concurrently pressed keys reaches zero. This way, it is possible to handle chords of multiple keys starting with the key specified in the <key> node. For the routing of global keys to clients, the order of <key> nodes is important. If multiple nodes exists for different labels, the first match will take effect. For example:
<global-keys> <key name="KEY_F11" label="launchpad -> testnit" /> <key name="KEY_F11" label="launchpad" /> </global-keys>
The "launchpad" client will receive all key sequences starting with F11 unless the "launchpad -> testnit" program is running. As soon as testnit gets started by launchpad, testnit will receive the events. If the order was reversed, launchpad would always receive the events.
Furthermore, the new version allows clients to dynamically allocate virtual framebuffers during the lifetime of the session. This solves two problems:
First, it enables a client to create a connection to nitpicker without donating much session quota in advance. The old interface required each screen-size-dependent client to donate as much memory as needed to allocate a screen-sized virtual framebuffer. For clients that are interested in the screen size but cover just a small portion of the screen (e.g., a banner, a menu, or an applet that sits in the screen corner), this over-provisioning is painful. The new interface allows such clients to upgrade the session quota for an existing session as needed.
Second, because each nitpicker session used to have a virtual frame buffer with a fixed size over the lifetime of the session, a client that wanted to implement a variable-sized window had to either vastly over-provide resources (by opening a session as large as the screen just in order to be prepared for the worst case of a maximized window), or it had to replace the session by a new one (thereby discarding the stacking order of the old views) each time the window changes its dimensions. The new interface accommodates such clients much better.
New terminal services
The new file terminal located at gems/src/server/file_terminal/ is a service that provides Genode's Terminal_session interface for a given file via a file-system session.
<config> <policy label="client1" filename="test.txt" /> <policy label="client2" filename="file.dat" io_buffer_size="4K"/> </config>
To keep things simple, a client can open only one file at the moment.
The new log terminal located at os/src/server/log_terminal/ forwards terminal output to a LOG session.
New C-runtime plugin for accessing block devices
Certain third-party code, which mostly originates from POSIX-like systems, uses normal file operations to access a block device from userspace , e.g. fsck(8) or mkfs(8). For this reason, access to a block device by using Genode's block-session interface through the normal C-runtime file operations is needed. The solution comes in the form of the new libc_block plugin. A program linked against this plugin obtains access to a block-session by opening the special device file "/dev/blkdev". Reading and writing from and to the block session - even on offsets that are not aligned to the block boundary - is handled by this plugin. For the actual program, the use of the plugin is completely transparent.
New file-system server for hybrid Genode/Linux systems
So far, scenarios depending on file accesses via the file-system session could not be run on Genode/Linux. But with the growing complexity of software that is ported to the framework, the demand for a full-fledged rapid-prototyping environment on Linux grows too. For example, our former approach to load required files as ROM modules stretches its limits with file sizes of several gigabytes. In the current release, we added a file-system server that provides transparent access to Linux files for Genode applications. As with other file-system servers, lx_fs can be configured to provide access to a specific subdirectory tree of the current working directory via a root policy entry.
<config> <policy label="client_1" root="/client_1_root" /> <policy label="client_2" root="/client_2_root" /> </config>
The provided service is comparable to "chrooting" a client. Accesses outside of the tree are denied by the server. Like the common practice with chroot, the client directory trees can be assembled by linking files into the visible directory tree. As we do not depend on hard links, the client tree may include links to entire sub-trees of the host file system via symbolic links to directories. Note that the current implementation does not support to create symlinks via the file-system interface.
Block-session extended by sync call
We extended the block-session interface with the ability to indicate that synchronization with the device is necessary. By now, all block device drivers in Genode work fully synchronous. Hence, there was no need for an explicit sync request by the client. When reworking block-device drivers to work asynchronously to potentially increase the performance or when adding a block cache component, the need for a synchronization call came up. Clients, like for instance a file system, have to ensure that their data is written back to the device at certain points of execution. The new sync() call enables block-server developers to implement their components asynchronously, while at the same time account for the integrity needs of block-session clients.
Configurable NIC buffer sizes for lwIP-based applications
Whenever a network client opens a NIC-session, it has to provide buffers for the receive/transmit (RX/TX) packet queues. By now, we've used relatively low default values for dimensioning those buffers. However, we identified that the demands by different applications vary a lot. Applications that require high throughput need large buffers whereas applications that issue sporadic network traffic can live with small buffers. Therefore, we extended the lwIP library's initialization with the ability to manually define the buffer sizes. Moreover, as lwIP is mostly used as network backend for Genode's libc, we extended the libc lwip plugin with a proper configuration option. To increase the default buffer sizes for a network application, the following snippet can be added to its configuration section
<libc rx_buf_size="1M" tx_buf_size="1M"/>
Support for recursive mutexes within the pthread library
As argued by one of the authors of the POSIX thread API, recursive mutexes are bad. Until now, we happily got away with not supporting them in Genode's pthread library. However, during our work on enabling QML on Genode, we hit the problem of ported 3rd-party code relying on recursive-mutex acquisition, which prompted us to finally implement the semantics as specified in the standard.
Device drivers
Raspberry Pi
We improved the platform support for the Raspberry Pi to a level where Genode's graphical demo scenario works well on the Pi. This required us to supplement device drivers for USB (for USB HID), Videocore mboxes (power management and HDMI), and for the framebuffer.
The USB host-controller driver is ported from the official fork of the Linux kernel for the Raspberry Pi. Unfortunately, it is not part of the normal Linux kernel. For this reason, we need to download it separately. There exists a prepare_rpi rule in the dde_linux/Makefile to automate this process.
The Raspberry-Pi-specific platform driver is used to access the features provided by the Videocore mboxes, i.e., power configuration and framebuffer setup. The framebuffer driver uses the platform interface to setup a screen mode of 1024x768.
The demo scenario at os/run/demo.run can be executed on the Raspberry Pi using the following steps:
-
Download the USB driver
make -C dde_linux prepare_rpi
-
Download the C library (needed for setjmp/longjmp as used by the USB driver)
make -C libports prepare PKG=libc
-
Create a build directory
./tool/create_builddir hw_rpi BUILD_DIR=build.rpi
-
Change to the new build directory
cd build.rpi
-
Enable the dde_linux and libports repositories by uncommenting the corresponding lines in your build directory's etc/build.conf file. Optionally, you may also enable parallelized building by adding the line
MAKE += -j4
-
Build the demo scenario
make run/demo
-
When the build process is finished, you can find the resulting ELF image at var/run/demo/image.elf. There are multiple options to boot the image on the Raspberry Pi, for example by using the u-boot boot loader or using JTAG. The simplest way, however, is to put an image directly on an SD-card. For doing so, prepare an SD-card with the regular firmware for the Raspberry Pi as found in the official Git repository at https://github.com/raspberrypi/firmware.
The first boot stage is executed by the so-called Videocore GPU, which fetches the boot code (called bootcode.bin) from the SD-card. At the second stage, the boot code loads the actual OS kernel from the SD-card (called kernel.img). Normally, this is the Linux kernel. To boot Genode instead of Linux, we have to persuade the boot code to load our image instead of the regular _kernel.img_ file. Because the Raspberry Pi boot code´ is not able to load ELF files, we first need to convert the Genode ELF image to a raw binary using the objcopy tool:
/usr/local/genode-gcc/bin/genode-arm-objcopy -Obinary \ var/run/demo/image.elf \ genode.imgFinally, we have to tell the boot code to load our image instead of the Linux kernel. This can be done by putting a simple config.txt file with the following content onto the SD-card:
kernel=genode.img kernel_address=0x00800000
|
|
Live from this year's Genode Hack'n'Hike: the graphical Genode demo running on the Raspberry Pi
|
Samsung Exynos 5
We extended the support for the Exynos-5 SoC with a new HDMI driver located at os/src/drivers/framebuffer/exynos5 as well as support for USB 3.0 mass storage. Thereby Genode reaches a pretty thorough driver coverage for the Exynos-5 platform including SATA 3.0 and 2.0, USB 3.0, DVFS, eMMC, networking, and HDMI.
To exercise those features, we have assembled an integrated system scenario for the Exynos-5-based Arndale board. The run script is located at ports-foc/run/l4linux_dynamic.run. It comes in two stages. At the first stage, the user can interact with the system over UART using the terminal_mux terminal multiplexer and a command-line interface. It is possible to start multiple instances of L4Linux and assign different block devices to the instances. It is also possible to run VIM using the Noux runtime environment to edit files on a VFAT-formatted SATA disk. The second stage can be started via the start graphical_demo. It presents another instance of the command-line interface. This instance allows the start of the GNU debugger or the scout demo program.
Libraries and applications
Qt5 with support for OpenGL and QML
The inclusion of Qt5 in Genode 13.08 apparently spawned interest in using QML as QML is widely regarded as the most important improvement of Qt5 compared with Qt4. With the current release, we are happy to announce the principal support for QML on Genode. The porting process was more elaborate than we had hoped for. First, the V8 Javascript engine as used by QML comes with its own platform abstraction (rather than building on top of Qt), which we had to implement for Genode. While doing so, we needed to complement our POSIX thread library to support recursive mutexes. Because QML relies on OpenGL, we had to integrate Qt with Genode's port of Mesa, which implied work on our EGL driver.
|
|
The QML based Qt5 example game called "samegame" running on Genode.
|
The QML support is still in an experimental state. It has been primarily developed and tested on Genode/Linux on x86_64. We plan to enable QML for the other Genode platforms and optimize performance during the upcoming release cycle.
If you want to start experimenting with QML on Genode right away, here are the basic steps for realizing a QML application:
-
The easiest way to get started is using a copy of the libports/src/app/qt5/qt_quicktest as a template.
-
main.cpp creates a QQuickView object, which loads the main QML file and starts to kick off the interpreter. Here, you may customize the name of the main QML file according to your needs.
-
The Qt resource file qt_quicktest.qrc contains QML files as well as data files as referenced by the QML code (such as images or Javascript files). All files required by your QML project must be listed here.
-
A copy of the QMake project file qt_quicktest.pro should be named after your project and customized according to your changes of the qrc file. Add further C++ sources and headers as needed.
-
The Genode build description file target.mk does not require any changes as long as your project does not depend on libraries other than Qt.
-
At libports/run/qt5_quicktest.run, you can find a suitable template for a Genode run script. You might need to adapt at least the program name and the RAM quota.
File systems based on FUSE
Besides a few special-purpose file systems (e.g., a RAM based file system and the TAR archive file system), up to now, Genode supports merely one general-purpose file system, namely FAT32, which is severely limited,
As one possible step to provide support for serious general purpose file systems in Genode, we took a look at the FUSE API, created an experimental implementation thereof, and ported fuse-exfat and fuse-ext2 to Genode. Since FUSE file systems implement file operations by using the actual path as an argument, it was easy to write a wrapper of these operations in form of a libc plugin called libc_fuse. By combing our FUSE implementation with the original FUSE file-system server code, accessing the file system in question from a program has become possible. As Genode does not use the normal mount mechanism employed by the original FUSE implementation, the start-up code, normally implemented in the main routine of a FUSE file-system server, needs to be provided by the FUSE file system port itself. The FUSE file system uses the new libc_block plugin described in Section New C-runtime plugin for accessing block devices to direct requests referring to a virtual Unix block device to a Genode block session.
For quickly trying out the new FUSE-based file systems, there are ready-to-use run scripts located at libports/run/libc_fuse_ext2.run and libc_fuse_exfat.run respectively. Before using those run scripts, make sure to prepare the packages "exfat" and "fuse-ext2" as found in the libports repository.
Launchpad
The launchpad demo application has been updated to use a more modern configuration syntax and has its built-in default configuration removed. Furthermore, launch entries have become able to supply configurations to sub systems, either via a <config> sub node or a <configfile> declaration.
DosBox
DosBox is a DOS emulator, which is primarily focused on running DOS games. Because it is fun to play with, we used DosBox as a proof of concept to demonstrate the ease of porting existing software to Genode and of course, we like to play as well. A step-by-step tutorial of how to approach such porting work will be published soon. For trying out DosBox on Genode, we added a simple-to-use run script at ports/run/dosbox.run.
Mesa / EGL driver
Our work on QML required us to improve our EGL driver for Mesa. Among other changes, we enabled the driver to let Mesa render into a user-provided buffer instead of the screen. This can be achieved with the eglCreateWindowSurface() function, which takes a buffer description as third argument.
libSDL improvements
For porting DosBox, we had to extend and improve our libSDL port. The former SDL_Timer implementation was too coarse. So we had to change it to a more fine granular one. In addition, we enabled the SDL_CDROM dummy code and ported SDL_net, which is needed for network support in DosBox.
Runtime environments
GNU Debugger
We improved the GDB support on Genode to cover advanced debugging features, namely the set var and call commands. As those commands require the debugger to modify the content of CPU registers of threads, we enhanced Genode's Cpu_session::state function such that it can do both obtain and modify the state of a thread while the thread is paused.
To make the use of GDB for debugging Genode subsystems more comfortable, we added a new command named "gdb" to Genode's command-line interface (cli_monitor). This command works similarly to the "start" command, but instead of starting the subsystem binary directly, an init subsystem gets started, which then starts terminal_crosslink, Noux, GDB and GDB monitor, which, in turn, starts the application binary as its target. The "gdb" command supports the following command line options:
- –ram
-
the initial RAM quota provided to the whole subsystem (including the GDB-related components)
- –ram-limit
-
limit for expanding RAM quota
- –gdb-ram-preserve
-
the RAM quota that GDB monitor should preserve for itself
For the "gdb" command to work, terminal_crosslink, noux, gdb_monitor and the file gdb_command_config are expected to be present as ROM modules. The GDB client needs to get mounted at /bin in Noux and the target binaries need to be available as ROM modules (loaded by GDB monitor) and also mounted at /gdb in Noux (loaded by the GDB client). Additionally, the source code of the target application can be provided at /gdb/src/ to Noux to enable source-level debugging. How the Noux mountings get established can be configured in the gdb_command_config file. The default configuration in os/src/server/cli_monitor/gdb_command_config mounts GDB from a tar archive named gdb.tar, the GDB target binaries come from a tar archive named gdb_target.tar, and the target source code comes from a tar archive named gdb_target-src.tar.
To simplify the assembly of such a scenario, the ports/run/noux_gdb.inc include file provides functions that help to create the needed tar files:
- create_gdb_tar
-
creates a tar archive for the GDB client
- create_binary_tar
-
creates a tar archive for the target application
- create_source_tar
-
creates a tar archive for the source code of the target application
- create_binary_and_source_tars
-
is a convenience wrapper for the previous two functions
The integration of GDB with cli_monitor is exemplified by the run script at ports/run/noux_gdb_dynamic.run.
Virtualization on NOVA
To ease the creation of custom virtual machine monitors on top of NOVA, we added a set of generic utilities to the public include location ports/include/vmm. As a nice side effect, this change simplifies the Genode-specific code of the Seoul VMM. In the future, the VMM utilities will ease the realization of alternative virtual-machine monitors on the NOVA hypervisor.
The Seoul VMM facilitates the co-location of the VMM and the guest OS within the same protection domain to keep context-switching costs as low as possible. However, this approach requires tight control over the virtual address space of the VMM. In particular, the VMM must not interfere with the guest-physical memory, which is always mapped at the lower part of the virtual address space. Maintaining this condition is complicated. Relaxing this rigid scheme would make our life much easier and would furthermore enable the use of shared libraries for the VMM. For this reason, we explored the option of running the guest OS in a distinct protection domain. This change does not only simplify the VMM, but it also nearly doubles the maximum configurable guest-memory size for a 32bit Genode/NOVA host system. The price for those benefits may be a degraded performance. However, using basic benchmarks (e.g., compiling the Linux kernel in a VM), we could hardly see negative effects. Co-location and non-co-location can be set by Seoul's new colocate configuration attribute.
ARM TrustZone
In Genode's release 12.11, we added experimental support for ARM TrustZone technology, which principally enabled the execution of Genode in the so-called secure world of TrustZone while executing Linux in the so-called normal world. These experiments were conducted on ARM's Versatile express platform. With the current release, we have added the i.MX53 SoC as additional platform to experiment with its TrustZone aware devices. If you want to try out a very basic example, starting a Linux instance running on the non-secure world beside Genode running in the secure one, you can test the tz_vmm.run run script on Versatile Express, i.MX53 Quickstart board, or i.MX53 SABRE tablet.
Platforms
Execution on bare hardware (base-hw)
The development of Genode's custom kernel platform for the ARM architecture goes full-steam ahead. The new version introduces a new way of using the kernel's asynchronous notification mechanism to deliver page faults and interrupts to the user land, yielding vastly simplified code - compared to the classical (L4) approach of providing a page-fault protocol via synchronous IPC. To better support real-time workloads, we enhanced the scheduler with static priorities. Finally, we complemented the base-hw platform with the code needed for process destruction. This way, we can execute dynamic workloads such as Noux. Base-hw is still at an experimental stage and very much in flux, but the pace of the current development illustrates well that it is making its way to become a fully-fledged base platform for Genode soon.
NOVA microhypervisor
After enabling multi-processor support and the destruction of kernel objects in the previous releases, the new implemented life-time management of kernel capability selectors represents the final piece of the puzzle to use NOVA as a fully-supported kernel platform for Genode.
A kernel capability selector on NOVA is similar in nature to a file descriptor on Unix, as it refers to a kernel object. In contrast to a file descriptor, however, the kernel object is not a file but the identity of an user-level object. Another difference is the way how a capability selector appears in a process. On Unix, most file descriptors are directly created by a process via system calls like open or socket. On NOVA, processes obtain capability selectors via IPC messages from other processes. Both file descriptors and capability selectors have in common that they should be closed when they are no longer needed. Otherwise, the name space of file descriptors or capability selectors gets polluted by stale kernel objects and long-running processes bear the risk of resource leakage.
Because Genode is implemented in C++, the framework is able to manage the lifetime of capability selectors by modelling the Capability type as a reference-counting smart pointer. After we first implemented this idea for the Fiasco.OC base platform, we have now successfully applied the same approach to the NOVA platform.
Fiasco.OC kernel and L4Linux
To enable the advanced features of the GNU debugger as described in Section GNU Debugger, we extended Genode's facility to access thread states such that the register contents of a paused thread (which is not currently executing a syscall) can get modified by the Cpu_session::state() function.
L4Linux is a paravirtualized Linux kernel that can be used on top of the Fiasco.OC kernel. To experiment with use cases where many instances of L4Linux are executed on a single machine, and as a way to exercise the new resource-balancing mechanisms described in Section Dynamic resource balancing, we extended L4Linux with a custom ballooning mechanism, which is similar to solutions like Xen's balloon driver. For this, the new parent interface extensions for requesting and yielding resources are used. L4Linux registers a yield signal context at its parent. Whenever the parent triggers a yield, the balloon driver "inflates its balloon", which means that it requests all pages available, and then frees the corresponding memory dataspaces.
The ballooning driver is enabled by default. From Genode's perspective, L4Linux behaves like a regular Genode subsystem that positively responds to resource-yield requests.
OKL4 kernel
We removed the support for the paravirtualized OKLinux kernel for OKL4. For running Linux on top of Genode, there exist several alternatives such as the Seoul VMM on NOVA, the ARM TrustZone support in base-hw, or the paravirtualized L4Linux kernel running on top of Fiasco.OC.