Skip to main content
Version: Developer

Sysbox Runtime

Overview​

The default Docker container runtime is runc. Sysbox is a free, open-source, drop-in replacement runtime originally developed by Nestybox. This guide installs Sysbox, configures Kasm Workspaces to launch sessions on it, and enables sudo for workspace users. As a result, users can install software, use root or sudo, and run system-level software such as systemd, buildx, and Docker, while the Kasm Agent stays protected against container breakout. When a workspace runs on Sysbox, it behaves much closer to a virtual machine, so users in a shared environment operate at root level with a lower risk of escaping isolation. For background on the security benefits, see the Sysbox User Guide: Security.

The following Kasm Workspaces core images, and all variants built on top of them, support a systemd init methodology and therefore run on the Sysbox runtime:

ContainerSource Code
kasmweb/core-almalinux-8:1.19.0Dockerfile
kasmweb/core-almalinux-9:1.19.0Dockerfile
kasmweb/core-debian-bullseye:1.19.0Dockerfile
kasmweb/core-debian-bookworm:1.19.0Dockerfile
kasmweb/core-kali-rolling:1.19.0Dockerfile
kasmweb/core-opensuse-15:1.19.0Dockerfile
kasmweb/core-oracle-8:1.19.0Dockerfile
kasmweb/core-oracle-9:1.19.0Dockerfile
kasmweb/core-parrotos-6:1.19.0Dockerfile
kasmweb/core-rockylinux-8:1.19.0Dockerfile
kasmweb/core-rockylinux-9:1.19.0Dockerfile
kasmweb/core-ubuntu-focal:1.19.0Dockerfile
kasmweb/core-ubuntu-jammy:1.19.0Dockerfile
note

When a Kasm Workspaces session uses the Sysbox runtime, it can take longer to become ready compared to a session launched with native Docker init. The delay comes from the general overhead of shiftfs and from running a full systemd init chain before the KasmVNC service starts.

Limitations of Kasm with Sysbox​

The following features are incompatible with Sysbox:

  • Persistent profiles.
  • NVIDIA GPU support within Kasm Workspaces.
  • Limited storage mapping or volume mapping support. No support exists between workspaces that use Sysbox and workspaces that do not. For details, see Map storage with Sysbox.

Prerequisites​

Before you begin, confirm the following:

  • A supported host operating system. Review the supported Linux distributions and the official installation guide.
  • A Linux kernel of at least 5.15, which is the minimum compatible kernel for Sysbox. Kasm recommends an Ubuntu Jammy host running the 6.5 LTS kernel, because that kernel integrates shiftfs. For a kernel between 5.15 and 6.3, follow Install shiftfs.
  • Administrator access to the Kasm Workspaces deployment, with permission to edit Workspaces and Group settings.
  • Root or sudo access on the Kasm Agent host to install packages and runtimes.

This guide does not replace the official Sysbox documentation. It documents the recommended setup for using the sysbox-runc runtime with Kasm Workspaces.

Solution approach​

This guide progresses through the following phases:

  1. Install the Sysbox runtime on the Kasm Agent host.
  2. Configure a workspace to use the Sysbox runtime.
  3. Enable sudo for workspace users.
  4. Map storage to Sysbox workspaces.

Detailed steps​

Install the Sysbox runtime​

Install Sysbox after Docker is installed but before Kasm Workspaces is installed. Sysbox can be installed on a host with an active Kasm Workspaces deployment, but Kasm cannot guarantee that containers will not be removed or deleted.

  1. Download and extract the Kasm installer so that the required Kasm dependencies install correctly. Navigate to the kasm_release directory and run the install_dependencies.sh script. This script installs all Kasm prerequisites, including Docker.

    cd /tmp
    curl -O https://kasm-static-content.s3.amazonaws.com/kasm_release_1.19.0.tar.gz
    tar -xf kasm_release_1.19.0.tar.gz
    sudo bash kasm_release/install_dependencies.sh

  2. Install the Sysbox runtime package.

    sudo apt-get install jq wget
    wget -O sysbox.deb https://github.com/nestybox/sysbox/releases/download/v0.6.7/sysbox-ce_0.6.7.linux_amd64.deb
    sudo apt-get install ./sysbox.deb
    tip

    For the most up-to-date Sysbox download link and for other architectures, refer to the official Sysbox GitHub releases page.

  3. Confirm that the Docker daemon was configured correctly.

    cat /etc/docker/daemon.json

    The output includes the sysbox-runc runtime:

    {
        "runtimes": {
            "sysbox-runc": {
                "path": "/usr/bin/sysbox-runc"
            }
        },
        "bip": "172.20.0.1/16",
        "default-address-pools": [
            {
                "base": "172.25.0.0/16",
                "size": 24
            }
        ]
    }

Configure a workspace to use the Sysbox runtime​

Any workspace that uses the Sysbox runtime must set its runtime and use systemd for init.

  1. Apply the following Docker Run Config Override (JSON) to the workspace.

    {
      "runtime": "sysbox-runc",
      "entrypoint": [
        "/sbin/init"
      ],
      "user": 0
    }

  2. Apply the setting at the scope you need. Set it per workspace under Admin > Workspaces > Edit > Docker Run Config Override (JSON). Set it at the group level under Access Management > Groups > Edit > Settings tab > Add Setting > run_config.

note

When a workspace uses the Sysbox runtime, it loses the ability to use an NVIDIA GPU through the NVIDIA Container Runtime. DRI3 with open-source drivers still functions in a Sysbox workspace. For details, see the DRI3 documentation.

Enable sudo for workspace users​

With Sysbox installed and set as the workspace runtime, enable sudo for the workspace.

  1. On the workspace configuration page, scroll to the Docker Exec Config block.

  2. Add the following JSON block. It runs as root when the workspace first launches, installs sudo, and adds kasm-user to the sudoers group. For more detail, see Installing and Configuring sudo via Docker Exec.

    {
     "first_launch":{
          "user":"root",
          "cmd":"bash -c '/usr/bin/desktop_ready && apt-get update && apt-get install -y sudo && echo \"kasm-user  ALL=(ALL) NOPASSWD: ALL\" >> /etc/sudoers'"
      }
    }

Map storage with Sysbox​

Important

When you use a Storage Provider or volume mapping configuration with Sysbox, attaching the same Storage Provider or volume mapping to a non-Sysbox container will cause permission problems between workspaces. The owner of the files and folders in a Sysbox container differs from the owner in a non-Sysbox container. The steps below explain why this happens and why it matters.

Using Kasm's Storage Provider or volume mapping with Sysbox changes how the feature normally works. For background, see the Sysbox documentation and the documentation on User Namespace ID Mapping.

When Kasm starts a workspace container normally, it runs the container as a user called kasm-user inside all of its workspaces. For details, see Building Custom Images. This user has a UID (User ID) and GID (Group ID) of 1000. When Kasm maps storage or volume drives to a workspace, the UID and GID of 1000 grant permission to the files and folders in the shared drive. For this reason, most of Kasm's Storage Mapping documentation uses a config similar to the following JSON:

{
     "driver" : "rclone",
     "driver_opts" : {
         "type" : "s3",
         "s3-provider" : "AWS",
         "s3-env-auth" : "false",
         "s3-region" : "us-east-1",
         "uid" : "1000",
         "gid" : "1000",
         "allow_other" : "true"
      }
}

In the JSON above, the "uid" : "1000" and "gid" : "1000" values set permissions on all files and folders in the share to match the kasm-user UID and GID. The workspace user then has permission to view and modify files and folders in the mapped volume. To adjust these values for Sysbox, you need to understand how Sysbox remaps UID and GID values and what Kasm does with them for file and folder permissions.

Sysbox changes permissions because it remaps the host UID and GID values to random, high values with no or very limited permissions on the host. Under the hood, the sysbox-runc runtime uses Docker's User Namespaces to handle this remapping. This allows apparent root access inside the container, while associating all container processes with a random high UID and GID value on the host.

Follow these steps to calculate the correct UID and GID for the storage or volume mapping.

  1. List the host users and their UID and GID values by reading /etc/passwd.

    $ cat /etc/passwd

    ...
    opc:x:1000:1000::/home/opc:/bin/sh
    ubuntu:x:1001:1001:Ubuntu:/home/ubuntu:/bin/bash
    kasm:x:1002:1002::/home/kasm:/bin/sh
    sysbox:x:1003:1003::/home/sysbox:/bin/false
    ...

    On this host, four users have login permissions, with UID values from 1000 to 1003 and GID values from 1000 to 1003.

  2. View the Sysbox remapped UID and GID values by reading /etc/subuid and /etc/subgid.

    $ cat /etc/subuid

    opc:100000:65536
    ubuntu:165536:65536
    kasm:231072:65536
    sysbox:296608:65536
    $ cat /etc/subgid

    opc:100000:65536
    ubuntu:165536:65536
    kasm:231072:65536
    sysbox:296608:65536
    note

    When using Sysbox CE, the only user responsible for running containers is the sysbox user. The only UID and GID values of importance in /etc/subuid or /etc/subgid are those associated with the sysbox user.

    In these files, Sysbox has remapped the sysbox user from a host UID of 1003 and host GID of 1003 to a random high UID of 296608 and a random high GID of 296608. It has also given the user a total of 65536 additional UID and GID values for any other required users inside its containers. From the host perspective, when a container runs on the sysbox-runc runtime, all processes and file or folder permissions associate with the UID and GID of 296608 on this host (this will likely be different on yours). From the container perspective, the user is root with a UID and GID of 0.

  3. Confirm the remapping by running a container on the Sysbox runtime and inspecting it from both the container and the host.

    Check Sysbox Runtime
    ## Run a container using the `sysbox-runc` runtime
    ubuntu@kasm-agent:~$ docker run --runtime=sysbox-runc --name=sysbox-test --rm -d alpine tail -f /dev/null
    2a70c22a9c3af66e3f4f76df3de9cf46dafed97ac7831953b0ebc3561eeeda91

    ## Container view using uid_map
    ubuntu@kasm-agent:~$ docker exec sysbox-test cat /proc/self/uid_map
            0     296608      65536

    ## User info from within the container
    ubuntu@kasm-agent:~$ docker exec sysbox-test id
    uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)

    ## Host process info of `tail` command running inside the container
    ubuntu@kasm-agent:~$ ps -fC "tail -f /dev/null"
    UID          PID    PPID  C STIME TTY          TIME CMD
    296608    171532  171503  0 21:35 ?        00:00:00 tail -f /dev/null

  4. Calculate the host UID of kasm-user. The container root user has a UID and GID of 296608. The kasm-user inside every Kasm Workspace has a UID and GID of 1000. Add 1000 to the Sysbox root UID of 296608 to get the host UID of kasm-user.

    296608 + 1000 = 297608
    Attention

    Aside from the kasm-user UID (1000), the values above are the ones the Sysbox service used for this deployment. Yours will likely be different.

  5. Update the storage or volume mapping values with the calculated UID and GID so file shares work correctly with these users.

    {
         "driver" : "rclone",
         "driver_opts" : {
             "type" : "s3",
             "s3-provider" : "AWS",
             "s3-env-auth" : "false",
             "s3-region" : "us-east-1",
             "uid" : "297608",
             "gid" : "297608",
             "allow_other" : "true"
          }
    }

Install shiftfs​

shiftfs is a kernel filesystem for the Linux kernel. It provides easier UID and GID shifting for containers and is a dependency of Sysbox. Linux kernels 5.15.x and later ship with shiftfs preinstalled. While kernels 5.15 to 6.3 in Ubuntu Jammy support shiftfs, install an up-to-date version for better performance.

  1. Build and install the shiftfs kernel module.

    sudo apt-get update
    sudo apt-get install -y make dkms git wget
    git clone -b k5.16 https://github.com/toby63/shiftfs-dkms.git shiftfs-k516
    cd shiftfs-k516/
    ./update1
    sudo make -f Makefile.dkms
    modinfo shiftfs

  2. Confirm the module loaded. The output looks similar to the following:

    filename:       /lib/modules/5.15.0-1031-aws/updates/dkms/shiftfs.ko
    license:        GPL v2
    description:    id shifting filesystem
    author:         Christian Brauner <christian.brauner@ubuntu.com>
    author:         Seth Forshee <seth.forshee@canonical.com>
    author:         James Bottomley
    alias:          fs-shiftfs

  3. Reboot the machine before continuing.

    sudo reboot

Common troubleshooting steps​

  • Workspaces take a long time to become ready. A session on the Sysbox runtime runs a full systemd init chain and uses shiftfs, which adds overhead compared to native Docker init. Allow extra startup time.
  • The workspace cannot use an NVIDIA GPU. NVIDIA GPU support through the NVIDIA Container Runtime is incompatible with Sysbox. Use DRI3 with open-source drivers, or run the workspace on the default runc runtime when GPU support is required.
  • File permission errors on mapped storage. Sysbox remaps the host UID and GID, so the kasm-user host UID equals the Sysbox root UID plus 1000. Read /etc/subuid and /etc/subgid, recalculate the value, and update the uid and gid fields in the mapping. Do not attach the same Storage Provider or volume mapping to both Sysbox and non-Sysbox workspaces.
  • The Docker daemon does not list the sysbox-runc runtime. Confirm that /etc/docker/daemon.json includes the sysbox-runc runtime path, then restart Docker.
  • shiftfs fails to load on a kernel between 5.15 and 6.3. Build and install the up-to-date shiftfs module, then reboot the host before launching workspaces.