How to use Genode with the NOVA hypervisor

When we started the development of Genode in 2006 at the OS Group of the TU Dresden, it was originally designated to be the user land of a next-generation and to-be-developed new kernel called NOVA. Because the kernel was not ready at that time, we had to rely on intermediate solutions as kernel platform such as L4/Fiasco and Linux during development. These circumstances led us to the extremely portable design that Genode has today and motivated us to make Genode available on the whole family of L4 microkernels. In December 2009, the day we waited for a long time had come. The first version of NOVA was publicly released:

Official website of the NOVA hypervisor: http://hypervisor.org

Besides the novel and modern kernel interface, NOVA has a list of features that sets it apart from most other microkernels, in particular support for virtualization hardware, multi-processor support, and capability-based security.

Why bringing Genode to NOVA?

NOVA is an acronym for NOVA OS Virtualization Architecture. It stands for a radically new approach of combining full x86 virtualization with microkernel design principles. Because NOVA is a microkernelized hypervisor, the term microhypervisor was coined. In its current form, it successfully addresses three main challenges. First, how to consolidate a microkernel system-call API with a hypercall API in such a way that the API remains orthogonal? The answer to this question lies in NOVA's unique IPC interface. Second, how to implement a virtual machine monitor outside the hypervisor without spoiling performance? The Vancouver virtual machine monitor that runs on top NOVA proves that a decomposition at this system level is not only feasible but can yield high performance. Third, being a modern microkernel, NOVA set out to pursue a capability-based security model, which is a challenge on its own.

Up to now, the NOVA developers were most concerned about optimizing and evaluating NOVA for the execution of virtual machines, not so much about running a fine-grained decomposed multi-server operating system. This is where Genode comes into play. With our port of Genode to NOVA, we contribute the workload to evaluate NOVA's kernel API against this use case. We are happy to report that the results so far are overly positive.

At this point, we want to thank the main developers of NOVA Udo Steinberg and Bernhard Kauer for making their exceptional work and documentation publicly available, and for being so responsive to our questions. We also greatly enjoyed the technical discussions we had and look forward to the future evolution of NOVA.

How to explore Genode on NOVA?

To download the NOVA kernel and integrate it with Genode, issue the following command from within toplevel directory:

 ./tool/ports/prepare_port nova

For the vesa driver on x86 the x86emu library is required and can be downloaded and prepared by invoking the following command from within the libports directory:

 ./tool/ports/prepare_port x86emu

For creating a preconfigured build directory prepared for compiling Genode for NOVA, use the create_builddir tool:

 <genode-dir>/tool/create_builddir nova_x86_32 BUILD_DIR=<build-dir>

This tool will create a fresh build directory at the location specified as BUILD_DIR. Provided that you have installed the Genode tool chain, you can now build the NOVA kernel via

 make kernel

For test driving Genode on NOVA directly from the build directory, you can use Genode's run mechanism. For example, the following command builds and executes Genode's graphical demo scenario on Qemu:

 make run/demo

Challenges

From all currently supported base platforms of Genode, the port to NOVA was the most venturesome effort. It is the first platform with kernel support for capabilities and local names. That means no process except the kernel has global knowledge. This raises a number of questions that seem extremely hard to solve at the first sight. For example: There are no global IDs for threads and other kernel objects. So how to address the destination for an IPC message? Or another example: A thread does not know its own identity per se and there is no system call similar to getpid or l4_myself, not even a way to get a pointer to a thread's own user-level thread-control block (UTCB). The UTCB, however, is needed to invoke system calls. So how can a thread obtain its UTCB in order to use system calls? The answers to these questions must be provided by user-level concepts. Fortunately, Genode was designed for a capability kernel right from the beginning so that we already had solutions to most of these questions. In the following, we give a brief summary of the specifics of Genode on NOVA:

We maintain our own system-call bindings for NOVA (base-nova/include/nova/) derived from the NOVA specification. We put the bindings under MIT license to encourage their use outside of Genode.
Core runs directly as roottask on the NOVA hypervisor. On startup, core maps the complete I/O port range to itself and implements debug output via the serial/UART I/O ports defined by the BIOS data area.
Because NOVA does not allow rootask to have a BSS segment, we need a slightly modified linker script for core (see src/core/core.ld). All other Genode programs use Genode's generic linker script.
The Genode Capability type consists of a portal selector expressing the destination of a capability invocation.
Thread-local data such as the UTCB pointer is provided by the new thread context management introduced with the Genode release 10.02. It enables each thread to determine its thread-local data using the current stack pointer.
NOVA provides threads without time called local execution contexts (EC). Local ECs are used as server-side RPC handlers. The processing time needed to perform RPC requests is provided by the client during the RPC call. This way, RPC semantics becomes very similar to function call semantics with regard to the accounting of CPU time. Genode already distinguishes normal threads (with CPU time) and server-side RPC handlers (Rpc_entrypoint) and, therefore, can fully utilize this elegant mechanism without changing the Genode API.
On NOVA, there are no IPC send or IPC receive operations. Hence, this part of Genode's IPC framework cannot be implemented on NOVA. However, the corresponding classes Ipc_istream and Ipc_ostream are never used directly but only as building blocks for the actually used Ipc_client and Ipc_server classes. Compared with the other Genode base platforms, Genode's API for synchronous IPC communication maps more directly onto the NOVA system-call interface.
The Lock implementation utilizes NOVA's semaphore as a utility to let a thread block in the attempt to get a contended lock. In contrast to the intuitive way of using one kernel semaphore for each user lock, we use only one kernel semaphore per thread and the peer-to-peer wake-up mechanism we introduced in the release 9.08. This has two advantages: First, a lock does not consume a kernel resource, and second, the full semantics of the Genode lock including the cancel-blocking semantics are preserved.
On the current version of NOVA, kernel capabilities are delegated using IPC. Genode supports this scheme by being able to marshal Capability objects as RPC message payload. In contrast to all other Genode base platforms where the Capability object is just plain data, the NOVA version must marshal Capability objects such that the kernel translates the sender-local name to the receiver-local name. This special treatment is achieved by overloading the marshalling and unmarshalling operators of Genode's RPC framework. The transfer of capabilities is completely transparent at API level and no modification of existing RPC stub code was needed.

Manually booting Genode on NOVA

NOVA supports multi-boot-compliant boot loaders such as GRUB, Pulsar, or gPXE. For example, a GRUB configuration entry for booting the Genode demo scenario with NOVA looks as follows, whereas genode/ is a symbolic link to the var/run/demo/genode directory created by invoking the demo run script.

 title Genode demo scenario
  kernel /hypervisor iommu serial
  module /genode/core
  module /genode/config
  module /genode/init
  module /genode/timer
  module /genode/nitpicker
  ...

Limitations

The current NOVA version of Genode is able to run the complete Genode demo scenario including several device drivers (PIT, PS/2, VESA, PCI) and the GUI. Still the NOVA support is not on par with some of the other platforms. The current limitations are: