Mounting Rootfs over 9P

The previous article focused on the U-boot setup, fetching the kernel using HTTP with the wget U-boot monitor command. The next step is configuring the kernel to mount the root file system over the network at start.

The mounting of the root file system is performed by Linux before executing the program that is specified by the init argument of the cmdline.

Linux has many features that can be enabled on a case by case basis depending on the requirements of a project. It has support for a vast amount of network protocols and file systems. The usual method of mounting a root file system over the network is achieved by using the NFSv4 protocol - It is the de facto standard when it comes to sharing files over the network on UNIX-like systems.

On Linux based systems, the NFS server is actually a kernel module, it is configured by modifying /etc/exports and restarting the nfs-kernel-server service.

User space NFS servers do exist, but they are not mainstream.

Mounting the rootfs using NFS also gets quirky due to the fact that by default the root user can not access an NFS mount point - this is by design. It can be bypassed with the use of the no_root_squash option.

For this article, I've decided to use the 9P protocol, it is less complex, but on the other hand also less stable and more exotic than NFSv4. In the Linux kernel, the 9P client is provided by the v9fs kernel module. https://github.com/v9fs

On the development host, I will be using diod https://github.com/chaos/diod a 9P file server which will host the root file system, diod is available as a package on Ubuntu and is easy to configure.

Building the kernel

Picking up in the project directory created in the previous article, add the Raspberry Pi fork of the stable kernel by adding it as a git submodule.

It can be achieved using multiple git commands:

git submoudle add --depth 1 https://github.com/raspberrypi/linux dependencies/linux
git -C dependencies/linux fetch origin stable_20250127 --depth 1
git -C dependencies/linux checkout FETCH_HEAD
git -C dependencies/linux branch rpi-6.6.y-stable FETCH_HEAD
git submodule set-branch -b rpi-6.6.y-stable dependencies/linux

This fetches the commit with the tag stable_20250127 without the git index history of all the commits, the tagged commit is then checked out and a local branch is created from it. Lastly, the submodule is updated to 'point' to this new local branch.

At first, I tried it with a single command:

git submodule add -b rpi-6.6.y --depth 1 https://github.com/raspberrypi/linux dependencies/linux

But it failed to checkout the branch rpi-6.6.y due to the presence of the --depth 1 argument. The --depth 1 can be omitted if you have a lot of disk space to spare and your network connection has a decent bandwidth.

But I live in the country side... 😀

We can now configure the kernel, be sure to also read the official documentation provided by RaspberryPi https://www.raspberrypi.com/documentation/computers/linux_kernel.html

cd dependencies/linux
KERNEL=kernel8
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig

This generates a default configuration for RPI3+ in ARM64 mode

By using menuconfig or by modifying the generated .config file modify the following Kconfig options, which by default are enabled and compiled as modules.

CONFIG_NET_9P=y
CONFIG_NET_9P_FD=y
CONFIG_9P_FS=y
CONFIG_9P_FSCACHE=y

This enables built-in support for the 9P protocol and filesystem, the default configuration for the RaspberryPI enables a great quantity of modules, depending on the use case most of them can be disabled, but this is beyond the scope of this article.

We will also have to configure the cmdline options accordingly:

CONFIG_CMDLINE="console=tty0 root=172.22.22.57 ip=dhcp rootfstype=9p rootflags=trans=tcp,version=9p2000.L,aname=/home/etag/src/rpi_boot/out/rootfs rw init=/bin/start"
CONFIG_CMDLINE_FORCE=y

Specifying a hard coded IP address of the development host, the transport type, version and most importantly the path of the 9P export.

Next, build the kernel and copy the resulting image to the webroot served by NGINX

make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j 8

cd - # Go back to top level project directory
cp dependencies/linux/arch/arm64/boot/Image out/kernel/current

Configuring diod

In the previous article, the web server was configured to serve the contents of the out/kernel directory, Now let's create out/rootfs which will contain the root file system, currently only a single bin directory.

mkdir -p out/rootfs/bin

diod can be installed by using the aptitude package manager:

apt-get install diod

The most simple way to configure diod is to create a configuration file in etc/diod.conf

auth_required = 0
exports = {
    { path="/home/etag/src/rpi_boot/out/rootfs", opts="noauth" },
}

Please note that there is no authentication yet - I will focus on security at later step.

Start diod:

sudo diod -c etc/diod.conf -S -U etag

The -S argument squashes all users to the user specified with the -U argument, which owns the exported directory.

Testing it

Since we have no user space environment yet, we need to supply a program that will be executed by the kernel

and become PID 1, this program can not use libc either since it usually comes with the environment.

I found a simple program here https://peterdn.com/post/2020/08/22/hello-world-in-arm64-assembly/

It writes 'hello world!' to fd 1 using a syscall, I've added an infinite loop to avoid a panic because PID 1 is not supposed to exit.

stub.S:

.data
/* Data segment: define our message string and calculate its length. */
msg:   
 .ascii        "Hello world!\n"
len = . - msg

.text
/* Our application's entry point. */
.globl _start

_start: 
    /* syscall write(int fd, const void *buf, size_t count) */
    mov     x0, #1      /* fd := STDOUT_FILENO */
    ldr     x1, =msg    /* buf := msg */
    ldr     x2, =len    /* count := len */
    mov     w8, #64     /* write is syscall #64 */
    svc     #0          /* invoke syscall */

.infloop: 
   b .infloop

Assemble it and create the write the resulting test executable to the bin directory of the rootfs.

aarch64-linux-gnu-as -o stub.o stub.S
aarch64-linux-gnu-ld -s -o out/rootfs/bin/start stub.o

Reboot the raspberryPI

Should give the following output on the screen:

[    8.166134] VFS: Mounted root (9p filesystem) on device 0:18
[    8.175286] devtmpfs: error mounting -2
[    8.187641] Freeing unused kernel memory: 4928K
[    8.194563] Run /bin/start as init process
Hello world!

Conclusion

The next article will focus on setting up a DHCP server and a DNS server to avoid hardcoding values in the configuration, which means that there will be 4 services running. So a service manager has to be configured to control the services.

The project will eventually be pushed on Github because from the previous article, I had to adjust the URL of the kernel in the U-Boot script. And set the user name to my user in the NGINX configuration to avoid permission errors.

Stayed tuned for next part!