Rust OSDev Operating System Development in Rust

This Month in Rust OSDev: July 2022

Published on (edited by @phil-opp)

Welcome to a new issue of "This Month in Rust OSDev". In these posts, we give a regular overview of notable changes in the Rust operating system development ecosystem.

This series is openly developed on GitHub. Feel free to open pull requests there with content you would like to see in the next issue. If you find some issues on this page, please report them by creating an issue or using our comment form at the bottom of this page.

rust-osdev Projects

In this section, we give an overview of notable changes to the projects hosted under the rust-osdev organization.

uefi-rs

Maintained by @GabrielMajeri, @nicholasbishop, and @phip1611

The uefi crate provides safe and performant wrappers for UEFI, the successor to the BIOS.

We merged the following changes in July:

x86_64

Maintained by @phil-opp, @josephlr, @Freax13, and @rybot666

The x86_64 crate provides various abstractions for x86_64 systems, including wrappers for CPU instructions, access to processor-specific registers, and abstraction types for architecture-specific structures such as page tables and descriptor tables.

In July, we merged the following changes:

Thanks to @Zildj1an for their contribution!

xhci

Maintained by @toku-sa-n

The xhci crate provides types of xHCI structures, such as Registers and TRBs.

We merged the following changes this month:

Thanks to @Demindiro and @ytoml for their contributions!

Call for Participation

Want to contribute to a Rust OSDev project, but don't know where to start? Help with one of these outstanding issues!

  • (rust-osdev/volatile) Various improvements for the new design

    We are currently discussing a new design for the volatile crate based on raw pointer types in order to avoid potential undefined behavior. The linked pull requests proposes an implementation where the volatile wrapper type is Copy and its methods take self by value. We haven't reached a decision yet, so if anyone has more input on this, please join the discussion. For more context, see also PR #22.

If you maintain a Rust project related to operating system development and are looking for contributors, especially for tasks suited to people getting started in this space, please create a PR against the next branch with the tasks you want to include in the next issue.

Other Projects

In this section, we describe updates to Rust OS projects that are not directly related to the rust-osdev organization. Feel free to create a pull request with the updates of your OS project for the next post.

rust-lang/rust

(Section written by @phil-opp)

This Month, @dvdhrm started an initiative to get the *-unknown-uefi targets to Tier-2. As a first step, they added a detailed description of the platform to the rustc book. The entry is already published in the nightly release of the book.

Please reach out if you would like to help with this!

Theseus OS

(Section written by Kevin Boos (@kevinaboos))

Theseus is a safe-language OS written from scratch in Rust that is in the process of migrating from pure academic research objectives to more general usability and legacy compatibility.

Theseus dynamically loads and links all components at runtime, and executes everything in a single address space and at a single privilege level. With this, Theseus employs intralingual design principles to maximally leverage the strengths of the Rust language, allowing the compiler to both view and semantically understand (and thus extend its safety checks to) all components from top-level apps to low-level kernel entities. Theseus strives to not only ensure isolation between different entity components and subsystems in the OS, but also to go further than language safety to provide various invariants that guarantee properties about most system-provided types. For more, check out the Theseus OS Book or our academic papers and presentations.

Theseus is fully open-source and always welcomes contributions from anyone.

Over the past month (or two), Theseus OS made significant progress on a variety of topics:

Check out the Theseus OS blog for the latest details.

nt-list: Windows Linked Lists in idiomatic Rust

(Section written by @ColinFinck)

On his quest to develop a ReactOS/Windows bootloader in Rust, Colin Finck released another building block as a reusable no_std crate this month. After nt-hive for reading Windows registry hive files and ntfs to access Microsoft's proprietary NTFS filesystem, the nt-list crate provides a type-safe and idiomatic Rust interface to work with Windows Linked Lists, known as LIST_ENTRY and SINGLE_LIST_ENTRY. This is what Windows, Windows drivers, and components influenced by Windows (e.g. UEFI) have been using for a long time to uniformly handle linked lists.

Colin's blog post goes into detail about some of the differences between textbook and Windows linked lists and the challenges in coming up with a safe Rust implementation. The final interface provided by nt-list is as simple to use as Vec while being fully compatible to the original LIST_ENTRY. The compatibility is proven in a WinDbg debugging session:

Using WinDbg to traverse a Windows Linked List created by the nt-list Rust crate

If you want to give it a spin, the crate is available on crates.io, and make sure to also check the docs.

Comparison between phip1611/simple-chunk-allocator and rust-osdev/linked-list-allocator

(Section written by @phip1611)

In March 2022, Philipp Schuster proposed his simple-chunk-allocator crate. It focuses on being a very simple-to-use general purpose allocator that "just works" for various workloads in no_std context. However, there are other allocators, such as rust-osdev/linked-list-allocator. When you choose an allocator, you should not only consider the API and how to set it up, but actually the runtime characteristics. OS research has shown us that there is no perfect allocator. You can optimize an allocator for speed, memory utilization (i.e., prevent fragmentation), code simplicity, and worst case execution time. There exist different strategies to reach those goals: first-fit, next-fit, best-fit

Recently, Philipp benchmarked his simple-chunk-allocator against rust-osdev/linked-list-allocator to learn under which conditions which performs better. But at first, let's point out some differences. simple-chunk-allocator needs a static backing storage for heap and an additional static backing storage for its internal bookkeeping. linked-list-allocator can solve this better by organizing the heap with the heap backing memory itself. simple-chunk-allocator uses a slightly adjusted variant of best-fit that tries to reduce fragmentation. linked-list-allocator is a first-fit allocator that has a lower performance to more heap is used.

The relevant outcome is that simple-chunk-allocator always outperforms linked-list-allocator in median allocation time. For stress tests with a low heap consumption, thus, no long stress test with 90% and more heap usage, simple-chunk-allocator also outperforms linked-list-allocator in average allocation time. However, if the heap is full and frequent allocations happen, the average (but not the median) allocation time of linked-list-allocator is better. Also, linked-list-allocator can come close to 100% heap usage which is not the case for simple-chunk-allocator, because it suffers from internal fragmentation under certain circumstances. Last but not least, even small allocations always takes up a multiple of the used chunk size in simple-chunk-allocator.

In the end, there is no optimal allocator. You must choose which properties are more relevant for your scenario. For concrete measurements, please head to the README of simple-chunk-allocator.

Join Us?

Are you interested in Rust-based operating system development? Our rust-osdev organization is always open to new members and new projects. Just let us know if you want to join! A good way for getting in touch is our Zulip chat.

Comments