• rv32-sail - high-level emulator for RV32IM 🔥

[[https://img.shields.io/badge/version-1.0pre-orange.svg]] [[https://img.shields.io/badge/license-MIT-blue.svg]]

Run it [[https://riscv.ls0f.pw][in your browser]], right now!

rv32-sail is a software emulator for a small computer that uses the [[https://risc-v.org][RISC-V]] instruction set. While other system emulators (such as [[https://www.qemu.org][QEMU]], or [[https://github.com/rems-project/sail-riscv][sail-riscv]]) are designed to boot systems like Linux or [[https://sel4.systems][seL4]], rv32-sail is more modest: it currently is designed to emulate something closer to an advanced microcontroller. (Think of it as an emulator for a system like the [[https://www.sifive.com/boards/hifive1][HiFive1]] -- not the [[https://www.sifive.com/boards/hifive-unleashed][HiFive Unleashed]].)

rv32-sail does not just come with the emulator: it's more like a full Software Development Kit, similar to some embedded microcontroller boards. It also comes with a handy environment containing a C cross compiler, the emulator, as well as a robust build system and test suite, and demonstrations. If you want to try RISC-V assembly programming, write your own simple "bare metal" programs, or extend the RISC-V instruction set, this might be for you!

#+BEGIN_QUOTE NOTE: rv32-sail ships a RISC-V software emulator, which is an approximate representation of the CPU as a sequential software program. It is not "cycle-accurate" and can not model clocks, digital logic, power or area consumption of a physical RISC-V CPU in any way! If you want to the Verilog equivalent of rv32-sail, try [[https://github.com/cliffordwolf/picorv32][picorv32]]! #+END_QUOTE

• Features

• High-level RISC-V ISA emulator, written in [[https://www.cl.cam.ac.uk/~pes20/sail/][Sail]]
  • Clean code is paramount with lots of comments.
  • High-level, type-safe, functional-imperative description of RISC-V.
  • Part hand-written (execution engine), part generated (decoder) at compile time.
  • Implements all of the base RV32IM instruction set as well as a select amount of CSRs. Machine (M) mode only, no User (U) mode support (yet).
• Robust build system, toolchain, programming environment.
  • Global dependencies (GCC, Haskell, Sail) are provided with [[https://nixos.org/nix][Nix]], so every dependency is taken care of for you, on almost any Linux distribution. Embedded programming toolchain nightmares are over, and you never fiddle with prerequisites -- and you can delete everything in an instant when you're done.
  • The build system is written using [[https://shakebuild.com][Shake]] -- it's fast and highly accurate, meaning you will (hopefully) never get out of date builds, broken builds, or build failures due to invalid dependency tracking.
  • Designed to be used incrementally and easy for contributors. Most of the Nix complexity is completely hidden, and the build system is easy as pie to use -- like a normal shell script.
  • Uses GitHub actions for continuous deployment of binary artifacts and examples (Docker, WebAssembly). This makes it easy for users to play.
  • When developing, you can download all the programming tools without compiling anything at all, thanks to [[https://cachix.org/][Cachix]]!
• Included programming tools
  • GCC 8.x with the RISC-V backend is the primary C toolchain.
  • A firmware shim library called libfirm provides the basics needed for setting up the environment -- initial stack allocation and calling your main function, etc.
  • Includes a very minimal libc that provides some of the core fundamentals for C programming, including basic standard headers, library functions etc.
  • Loads raw ELF binaries that can be booted and executed directly.
• Included demo code
  • Full test firmware based on [[https://github.com/cliffordwolf/picorv32][picorv32]] firmware, and upstream RISC-V assembly tests, for basic verification.
  • Benchmarks: both [[https://en.wikipedia.org/wiki/Dhrystone][Dhrystone]] and [[https://www.eembc.org/coremark/][CoreMark]] are built by default.
  • Fun: a full prototype FORTH implementation, based on JonesForth

Note that rv32-sail is not designed to be packaged for Linux distributions, or used like -- or as an alternative to -- a real emulator such as QEMU. It is quite slow, and it is designed for correctness and as the basis for software stack development, with design flexibility in the simulator.

The end goal of this project is to implement a full, robust emulator for an embedded RISC-V CPU with [[https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/][CHERI extensions]], based on the paper /[[_][CheriRTOS: A Capability Model for Embedded Devices]]/, as well as an accompanying capability-first RTOS. (If we want to stir up our wild imaginations, such a device could be a pure open hardware/FOSS replacement for devices like the ESP32.)

• Table of Contents :TOC_4_gh:

• [[#rv32-sail---high-level-emulator-for-rv32im-][rv32-sail - high-level emulator for RV32IM 🔥]]
• [[#features][Features]]
• [[#quickstart][Quickstart]]
  • [[#quickstart-in-your-browser][Quickstart: In your browser]]
  • [[#quickstart-docker][QuickStart: Docker]]
  • [[#quickstart-nix-with-cachix][QuickStart: Nix with Cachix]]
• [[#building][Building]]
• [[#usage][Usage]]
  • [[#running-demos-and-tests][Running demos and tests]]
    • [[#smoke-test-firmware][Smoke-Test firmware]]
    • [[#dhrystone-and-coremark-benchmarks][Dhrystone and CoreMark benchmarks]]
    • [[#forth][FORTH]]
  • [[#basic-emulator-options][Basic emulator options]]
  • [[#emulator-configuration-options][Emulator configuration options]]
  • [[#cleaning-up][Cleaning up]]
• [[#source-code-layout][Source code layout]]
• [[#softwarefirmware-development][Software/Firmware Development]]
  • [[#programming-toolchain-support][Programming toolchain support]]
    • [[#cc][C/C++]]
    • [[#rust][Rust]]
  • [[#libfirm-starter-guide][Libfirm starter guide]]
  • [[#adding-your-own-programs][Adding your own programs]]
• [[#authors][Authors]]
• [[#license][License]]

• Quickstart

If you just want to try out the emulator and the demos at no cost, there are three easy ways to do it: your Browser, Docker, and a [[https://nixos.org][Nix]] Cache.

** Quickstart: In your browser

As mentioned above, you can run the emulator [[https://riscv.ls0f.pw][in your browser]], right now. This requires a modern system with support for [[https://webassembly.org][WebAssembly]]. It has a neat retro UX, using the JavaScript [[https://hterm.org][hterm]] library. It offloads all of the compiled code into a [[https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers][Web Worker]] so the main UI thread isn't blocked, even when running "compute intense" demos like CoreMark.

This page is automatically updated on every commit to the master branch. It has been tested on recent Chrome (Desktop), and Safari (iOS), though mobile input is quite glitchy right now. Firefox seems to have a WebAssembly loading glitch that requires cache purges, and hterm integration problems; I am not sure what extent this is inherent or just my code.

Note: The WebAssembly build is very unstable, moreso than the ordinary emulator -- so don't be surprised if very basic functionality doesn't work. Patches to help Sail WebAssembly support, hterm integration, and emscripten updates are very welcome.

** QuickStart: Docker

There's a Docker Container that's automatically compiled and built (using [[https://github.com/features/actions][GitHub Actions]]) on every commit to the master branch. You can try the emulator immediately by running one of the included demos:

#+BEGIN_SRC language docker run --rm thoughtpolice/rv32-sail:master -e smoke.elf docker run --rm thoughtpolice/rv32-sail:master -e dhrystone.elf #+END_SRC

All of the demos in the package (Dhrystone, CoreMark, etc), as well as the smoke test firmware (smoke.elf) are included in the Docker container.

#+BEGIN_QUOTE

Note: The :master tag for the Docker repository is always updated when the master branch is, etc etc. There is also a tag corresponding to every git commit, and the Nix package version (e.g. 1.0 or 1.0pre_...)

The :latest tag is reserved for stable releases -- there are no stable releases as of now. #+END_QUOTE

** QuickStart: Nix with Cachix

There's a [[https://cachix.org][Cachix]] Cache containing all the build tools as well as pre-built copies of the emulator, updated on every commit to the master branch. Read the "Building" section below to get started.

• Building

#+BEGIN_QUOTE NOTE: You must* have [[https://nixos.org/nix][the Nix Package Manager]] installed in order to build the emulator! While you can install Sail and the dependencies yourself, this is the only supported development method, and the only environment that tests, builds etc are run with, in the main repository. Should you ignore this, I cannot help you with any issues, and you should not be surprised if things don't work!

If you are running an x86_64 Linux distribution, using systemd, and capable of executing sudo on your machine, then you can do a quick installation easily from your shell:

#+BEGIN_SRC bash $ sh <(curl https://nixos.org/nix/install) --daemon #+END_SRC

(Please read the shell script. All it does is download a tarball and execute an actual installation script inside, and you can find [[https://github.com/NixOS/nix/tree/master/scripts][the source code for those scripts]] as well.)

You must pass the --daemon flag to the installer! (If you don't, you will not have sandboxing support enabled in Nix, which could lead to non-reproducible builds, and other strange build failures.)

This should work on any modern Linux distribution with namespace support and systemd as the init system. Then you can log back into your user account -- nix, nix-shell and other tools will now be available.

In the future, I hope to also provide static binary distributions containing the emulator and test firmware, too. #+END_QUOTE

Currently, the primary way to compile the emulator and firmware is to use the build system is by simply invoking it directly using the bake.hs script. This is designed to be as easy as possible. If you have a recent version of Nix installed, this will essentially "just work" -- though the first invocation will take some time (see below).

There is a binary cache available thanks to [[https://cachix.org][Cachix]] which contains copies of all the build tools and builds from the master branch. First, as root or someone who can sudo, add yourself to trusted-users, and restart the nix-daemon service:

#+BEGIN_SRC bash grep -q "trusted-users.*$USER" /etc/nix/nix.conf ;
[ ! $? -eq 0 ] && echo "trusted-users = root $USER" | sudo tee -a /etc/nix/nix.conf sudo systemctl restart nix-daemon.service #+END_SRC

Then you can configure binary caches with a local file in $HOME/.config/nix/nix.conf:

#+BEGIN_SRC bash mkdir -p "$HOME/.config/nix" cat > "$HOME/.config/nix/nix.conf" <<EOF substituters = https://cache.nixos.org https://aseipp.cachix.org trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= aseipp.cachix.org-1:UaUH2DczaM7ytMZlyuHtpYq8FAziIQnjmGMaB45rvRw= EOF #+END_SRC

Now, clone the repository and build everything:

#+BEGIN_SRC bash git clone https://github.com/thoughtpolice/rv32-sail cd rv32-sail/ nix build --no-link -f release.nix # download cached binaries ./bake -j # do a local build #+END_SRC

Provided this all works correctly, you won't have to build any toolchains locally, you can simply download everything on demand, provided you have a few GB of space free (download sizes are much smaller than that).

If you want to control or invoke the underlying Sail toolchain directly (for example, to pass different options, or examine the build environment), simply run nix-shell instead:

#+BEGIN_SRC $ nix-shell ...

[nix-shell:~/sail-riscv32]$ sail -v ...

[nix-shell:~/sail-riscv32]$ riscv32-unknown-elf-gcc --version ... #+END_SRC

Once you're inside nix-shell, you can also run the bake command, which is an equivalent method to run the bake.hs script, with all the same arguments:

#+BEGIN_SRC [nix-shell:~/sail-riscv32]$ bake -j Build completed in 0.01s #+END_SRC

This command is not only shorter to type, but it executes faster than the "normal" shell script. For iterative development, you may find having an extra terminal or tmux window where you run bake quite useful!

• Usage

Once you've built the emulator and test/demo firmware, all of those artifacts will be available under the ./build directory.

** Running demos and tests

*** Smoke-Test firmware

The self-testing firmware is available under ./build/t/smoke.elf, and can be loaded immediately. At the end, the emulator will spit out some runtime statistics, as well as a register dump:

#+BEGIN_SRC ./build/cruise -e build/t/smoke.elf [Sail] Allocating new block 0x0 [Sail] ELF Initial PC: 0x0 [Sail] Executing reset vector...

RUNNING RISC-V TESTS

...

FINISHED RISC-V TESTS

Sieve test: 1st prime is 2. 2nd prime is 3. 3rd prime is 5. ... 31st prime is 127. checksum: 1772A48F OK

CPU stats: Cycle Counter: ... Instruction Counter: ... CPI: ...

DONE [Sail] Trap (EBREAK) encountered - exiting [Sail] Finished! [Sail] Register dump: x0: 0x00000000 ra: 0x000000A0 sp: 0x00010000 gp: 0xDEADBEEF tp: 0xDEADBEEF t0: 0x0000018C t1: 0x0000002A t2: 0x00000000 fp: 0x00000000 s1: 0x00000000 a0: 0x0000002A a1: 0x0000000A a2: 0x00000005 a3: 0x00000000 a4: 0x00000030 a5: 0x0000000A a6: 0x00000000 a7: 0x00000000 s2: 0x00000000 s3: 0x00000000 s4: 0x00000000 s5: 0x00000000 s6: 0x00000000 s7: 0x00000000 s8: 0x00000000 s9: 0x00000000 s10: 0x00000000 s11: 0x00000000 t3: 0x00000000 t4: 0x00000000 t5: 0x00000000 t6: 0x00000000

[Sail] Executed Instructions: ... [Sail] Nanoseconds Elapsed: ... [Sail] Approximate IPS: ... #+END_SRC

This will:

• Boot up the CPU, and jump to the initial reset vector (0x00000000), inside of src/boot,
• Jump to the main entry point defined in src/t/firmware/main.S,
• Run the RISC-V assembly language tests for RV32IM
• Run a demonstration of a Sieve to compute primes,
• Print some timer information (extracted from CSRs)

*** Dhrystone and CoreMark benchmarks

[[https://en.wikipedia.org/wiki/Dhrystone][Dhrystone]] and [[https://www.eembc.org/coremark/][CoreMark]] are included as demos; just do:

#+BEGIN_SRC ./build/cruise -e ./build/demos/dhrystone.elf ./build/cruise -e ./build/demos/coremark.elf #+END_SRC

*** FORTH

An old implementation of Forth I wrote, based on JonesForth, targeting the HiFive1. It's still incomplete -- try helping out!

#+BEGIN_SRC ./build/cruise -e ./build/demos/forth.elf #+END_SRC

** Basic emulator options

The three primary options you may use are:

• -l the cycle limit, which controls how many CPU cycles the emulator will execute before yielding. By default, the cycle limit is unlimited and the only way to terminate the emulator is through an EBREAK.
• -e the elf binary to load. Self explanatory.
• -C the configuration option spec; this allows you to set arbitrary integer/boolean values, controlling various CPU options. (See below.)

** Emulator configuration options

The emulator has several configuration options which can be set at runtime in order to control various aspects of the machine's behavior. Notably, this includes things like disabling certain instruction set features, etc. (These are hardware configuration settings that take priority over the MISA CSR; you can have a feature enabled and disable it later, etc.)

To get the list of options, invoke the cruise executable with the arguments -C help:

#+BEGIN_SRC ./build/cruise -C help #+END_SRC

** Cleaning up

You can completely delete this directory later at any time if you want to clean things up, or run bake clean to have it done for you.

• Source code layout

The primary directories you need to understand are:

• ./src/mk, which contains the build system, written in Haskell. This also includes the instruction decoder, which generates Sail code at build time to parse and pretty-print RISC-V instruction encodings.
• ./src/spec, which contains all the Sail code for the specification, including the execution engine.
• ./src/boot, which contains the initial bootloader shim and reset vector setup, written in assembly. (It's contained here so it's easier to find and read.)
• ./src/libfirm, a simple libc and bare-metal programming library that demos, tests, etc all share and use.
• ./src/t, which contains all the tests.
• ./src/demos, which contains a bunch of fun demo programs.
• ./nix, and release.nix, which contain the Nix code for provisioning all the needed tools.
• /.github contains all the code for this repositories' GitHub Actions workflows and commands, including tools for pushing Docker, HTML pages and Cachix uploads.

Everything else falls outside the primary raidus of the blast zone.

• Software/Firmware Development

** Programming toolchain support

*** C/C++

The default toolchain is a GCC 8.x RISC-V cross compiler for bare-metal targets using C.

C++ is currently not supported, but this is only due to a few missing runtime bits inside libfirm. Patches welcome.

LLVM is not supported. While the RISC-V LLVM backend continues to be upstreamed in various pieces, it (to my knowledge) is still quite unstable. In the future, we should ideally be able to use a copy of the upstream Nix LLVM package with the RISC-V backend enabled, and have Clang act as a cross compiler instead.

If adding a fork of LLVM/Clang with RISC-V support using Nix to build it is not too burdensome, it might be acceptable in the mean time.

In the long run, the plan is to ship fully equipped GCC and Clang RISC-V cross compilers, with C++ support.

*** Rust

As a result of the incomplete LLVM RISC-V support, Rust is also not supported.

If adding a fork of LLVM/Clang/Rust/ with RISC-V support using Nix to build it is not too burdensome, it might be acceptable in the mean time.

In the long run, the plan is to find a way to ship a bare metal Rust Nightly cross compiler, once RISC-V support in LLVM stabilizes.

** Libfirm starter guide

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec hendrerit tempor tellus. Donec pretium posuere tellus. Proin quam nisl, tincidunt et, mattis eget, convallis nec, purus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Nulla posuere. Donec vitae dolor. Nullam tristique diam non turpis. Cras placerat accumsan nulla. Nullam rutrum. Nam vestibulum accumsan nisl.

** Adding your own programs

Nunc rutrum turpis sed pede. Nullam eu ante vel est convallis dignissim. Nunc porta vulputate tellus. Donec vitae dolor. Vivamus id enim.

• Authors

rv32-sail is the result of many pieces of code being stitched together, both my own, and others. There are many various people and projects who have contributed to its birth. See [[https://raw.githubusercontent.com/thoughtpolice/rv32-sail/master/AUTHORS.txt][AUTHORS.txt]] for the list of contributors to the project and some notes on who was responsible.

• License

MIT, but, as noted, much of the code is authored by others and, thus, available under various terms. (Almost all of these being extremely free non-copyleft licenses, including ASL2.0, BSD3, MIT, ISC, and Public Domain.) See [[https://raw.githubusercontent.com/thoughtpolice/rv32-sail/master/COPYING][COPYING]] for precise terms of copyright and redistribution.