DEV Community

Sendil Kumar
Sendil Kumar

Posted on • Edited on • Originally published at sendilkumarn.com

WASI - WebAssembly System Interface with Wasmtime

#rust #webassembly #beginners #tutorial

WebAssembly enables running native code in the JavaScript engine. The compiled and optimised binary ensures better and consistent performance. The JavaScript engine provides the necessary runtime to execute the binary.

What if we bring the performance and portability of WebAssembly outside JavaScript execution environments? The answer is WASI.

Check out my book on Rust and WebAssembly here

WASI or WebAssembly System Interface is a system interface for the WebAssembly platform. WASI will enable running WebAssembly application on any Operating System or architecture provided that we have the runtime. Conceptually, this is similar to JVM. If you have a JVM installed then you can run any Java-like languages on it. Similarly, with a runtime, you can run the WebAssembly module.

It's an API designed by the Wasmtime project that provides access to several operating-system-like features, including files and filesystems, Berkeley sockets, clocks, and random numbers, that we'll be proposing for standardisation.

It's designed to be independent of browsers, so it doesn't depend on Web APIs or JS, and isn't limited by the need to be compatible with JS.

It has integrated capability-based security, so it extends WebAssembly's characteristic sandboxing to include I/O.

WebAssembly System Interface is the next step in WebAssembly's journey.

The way WASI works is simple. You write your application in your favourite languages like Rust, C or C++. Then build and compile them into WebAssembly binary targeting WASI environment. The generated binary requires a special runtime to execute. The runtime provides the necessary interfaces to the system calls.

To see WASI in action, we need a runtime. There are two different runtimes available (and hopefully many will be available later). They are

  • wasmtime
  • Lucet

WASI provides portability. It provides an option where you write once and run anywhere. Let us see wasmtime in action.

wasmtime - runtime for WebAssembly

Wasmtime is a standalone wasm-only optimizing runtime for WebAssembly and WASI. It runs WebAssembly code outside of the Web and can be used both as a command-line utility or as a library embedded in a larger application. Install the wasmtime runtime to run the WebAssembly binary.

The simplest way to install the wasmtime is by running the following command:

$ curl https://wasmtime.dev/install.sh -sSf | bash
$ ./wasmtime --version
wasmtime 0.9.0
Enter fullscreen mode Exit fullscreen mode

You can use --help option to list down various options available in the wasmtime command.

$ ./wasmtime --help
wasmtime 0.9.0
Wasmtime WebAssembly Runtime

USAGE:
    wasmtime <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    config      Controls Wasmtime configuration settings
    help        Prints this message or the help of the given subcommand(s)
    run         Runs a WebAssembly module
    wasm2obj    Translates a WebAssembly module to the native object file
    wast        Runs a WebAssembly test script file

If a subcommand is not provided, the `run` subcommand will be used.

Usage examples:

Running a WebAssembly module with a start function:

  wasmtime example.wasm

Passing command-line arguments to a WebAssembly module:

  wasmtime example.wasm arg1 arg2 arg3

Invoking a specific function (e.g. `add`) in a WebAssembly module:

  wasmtime example.wasm --invoke add 1 2
Enter fullscreen mode Exit fullscreen mode

We have successfully installed the wasmtime. Now we execute the WebAssembly binary code in the runtime provided by the wasmtime.

To compile the native code applications into WASI compatible, the Rust ecosystem provides wasm32-wasi target. This target is available in the nightly version and then install the target using rustup.

$ rustup target add wasm32-wasi --toolchain nightly
Enter fullscreen mode Exit fullscreen mode

Then we use cargo to build the application for this target using

$ cargo +nightly build --target wasm32-wasi
Enter fullscreen mode Exit fullscreen mode

It is time to takewasmtime for a spin.

How it works...

Let us create a new Rust application using Cargo. To create a new application let us run the following command

Lazy to write the code - check out the repository here:

GitHub logo sendilkumarn / sizer

WASI demo repo 

$ cargo new --bin sizer
Enter fullscreen mode Exit fullscreen mode

This will create a binary application. We can run the application using

$ cargo run 
Hello, world!
Enter fullscreen mode Exit fullscreen mode

Now change the src/main.rs with the following contents:

use std::{env, fs};

fn process(current_dir: &str) -> Result<(), String> {
    for entry in fs::read_dir(current_dir).map_err(|err| format!("{}", err))? {
        let entry = entry.map_err(|err| format!("{}", err))?;
        let path = entry.path();
        let metadata = fs::metadata(&path).map_err(|err| format!("{}", err))?;
        println!(
            "filename: {:?}, filesize: {:?} bytes",
             path.file_name().ok_or("No filename").map_err(|err| format!("{}",     err))?,
         metadata.len()
         );
     }
 Ok(())
}

fn main() {
    let args: Vec<String> = env::args().collect();
    let program = args[0].clone();
    if args.len() < 2 {
        eprintln!("{} <input_folder>", program);
        return;
     }
    if let Err(err) = process(&args[1]) {
        eprintln!("{}", err)
    }
}
Enter fullscreen mode Exit fullscreen mode

Given a directory, the process function runs through the directory and list all the files and their size information. The above code has two functions, main and process function. The main function is called first. The function validates whether we provide the directory information as an argument. If the length of arguments is less than 2, it throws an error. If you have provided the argument, then it calls the process function.

The process function takes in the current directory argument. Then it reads the folder for any files available and then it prints out the files that are in the folder and the file sizes.

$ cargo run ./
 filename: "Cargo.toml", filesize: 237 bytes
 filename: "target", filesize: 128 bytes
 filename: "Cargo.lock", filesize: 136 bytes
 filename: ".gitignore", filesize: 8 bytes
 filename: ".git", filesize: 288 bytes
 filename: "src", filesize: 96 bytes
Enter fullscreen mode Exit fullscreen mode

We will create the WebAssembly module using the following command:

$ cargo +nightly build --target wasm32-wasi
 Compiling sizer v0.1.0 (/some/path/to/folder/sizer)
 Finished dev [unoptimized + debuginfo] target(s) in 0.36s
Enter fullscreen mode Exit fullscreen mode

Now let us run the WebAssembly generated using wasmtime.

$ wasmtime target/wasm32-wasi/debug/sizer.wasm
 sizer.wasm <input_folder>
Enter fullscreen mode Exit fullscreen mode

Now let us pass the input argument to the runtime.

$ wasmtime target/wasm32-wasi/debug/sizer.wasm ./
 failed to find a preopened file descriptor through which "./" could be opened
Enter fullscreen mode Exit fullscreen mode

As we can see the wasmtime it does not have any access to the folder specified. It complains that it cannot find any file descriptor. By default, the wasmtime does not have any global permissions. Permission to view other directories and process that might be running in the Operating System. To give the wasmtime permission to access folder, we can provide the directory using --dir flag. The dir flag will create the necessary file descriptors that will enable the wasmtime to access the folder provided.

$ wasmtime target/wasm32-wasi/debug/sizer.wasm --dir=/path/to/sizer/folder /path/to/sizer/folder
filename: "Cargo.toml", filesize: 225 bytes
filename: "target", filesize: 192 bytes
filename: "Cargo.lock", filesize: 137 bytes
filename: ".gitignore", filesize: 19 bytes
filename: ".git", filesize: 288 bytes
filename: "src", filesize: 96 bytes
Enter fullscreen mode Exit fullscreen mode

Now the above command will print the expected results. WASI uses capabilities model for security. Read more about capabilities security model here.

The wasmtime provides a wrapper API over the system calls. They are inspired and derived from POSIX systems. But the WASI core differs from POSIX in the following ways.

  • The WASI core has no processes in them. Processes provide no cleaner way to provide forks and execution in Operating Systems. Most of the Operating Systems uses processes to implement forks, execution that is complicated and difficult to maintain. Using processes in WebAssembly System Interface will make the application stick to Operating Systems process boundaries.

The APIs that WASI provides is still under active development. But it is important to note that the APIs provided by WASI is different from POSIX APIs. The former uses no processes and the security model is different. But POSIX APIs uses processes and the security model.

  1. The WASI API currently only supports blocking API calls.

  2. The WASI API currently does not support async.

  3. The WASI API does not have true "mmap" support.

This will be changing shortly once we churn out more APIs.

How to do it

The WASI APIs are named under the wasi_ namespace. For example, all the code related to file descriptors will be placed under the wasi_fd namespace.

For example in the last recipe, we have used __wasi_fd_filestat_get(). This API returns the attribute of the given file.

When working with the languages like Rust we will not need to worry about this low-level API calls. Since most of the Rust calls are converted into necessary wrapper APIs. But we will need to use these APIs when we try to debug or write WebAssembly Module by hand.

Let us go and create a WebAssembly Text format and use the WASI APIs straight away. We will createwriter.wat:

$ touch writer.wat
Enter fullscreen mode Exit fullscreen mode

Let us open the file in our favourite editor.

We will first define the base module for the WebAssembly Text Format.

(module )
Enter fullscreen mode Exit fullscreen mode

We will then have to import the WASI function such that we can call the function inside the WebAssembly Text Format. To import the WASI function:

(module
 (import "wasi_unstable" "fd_write" (func $fdw (param i32 i32 i32 i32) (result i32))
)
Enter fullscreen mode Exit fullscreen mode

Then we call the function inside the WebAssembly module using call.

Note that the API that we are calling and that is defined in the spec is slightly different. The APIs defined in the spec is of the format __wasi_fd_write but we have imported the fd_write function from the wasi_unstable namespace.

But before that, we need to define the memory and data that we will need to use.

(module
 ;; define the imported function
 (memory 1)
 (export "memory" (memory 0))

 (data (i32.const 12) "Hooray it's WASI\n")
)
Enter fullscreen mode Exit fullscreen mode

Here we just define the memory and export it. Then we create data with an offset of 12. This means that the data that we represent will be at the linear memory after an initial offset of 12 bytes.

Unlike, WebAssembly Modules for the Web, here the start function is not optional. So we will have to define the start function.

(module
 ;; define the imported function
 ;; define memory and data
     (func $main (export "_start")
          (i32.store (i32.const 0) (i32.const 12))
          (i32.store (i32.const 4) (i32.const 20))
          (call $fdw
                (i32.const 1)
                (i32.const 0)
                (i32.const 1)
                (i32.const 20)
           )
      drop
    )
)
Enter fullscreen mode Exit fullscreen mode

The main function (func main) stores the data from the data pointer. These pointers are then used to represent the iov_base and iov_len.

struct iovec {
      void  *iov_base;    /* Starting address */
      size_t iov_len;     /* Number of bytes to transfer */
};
Enter fullscreen mode Exit fullscreen mode

Then we call the fd_write method with the required four arguments. The arguments are the file descriptor, the pointer to the iov_base and iov_len and finally the new memory location in which it has to write the contents.

Finally, we discard the written bytes from the top of the stack.

How it works...

The wasmtime provides a way to use the WebAssembly Text format. So we do not need to worry about converting the WebAssembly Text Format into WebAssembly Module.

$ wasmtime writer.wat
Hooray It's `WASI`
Enter fullscreen mode Exit fullscreen mode

The wasmtime interpreter first validates whether the file provides is in the correct formation. Once the validation is successful, the 1 will compile the application. During this phase, the wasmtime compiler creates the binary code that will initiate the system call for the underlying architecture.

Finally, the drop will run and throw off some bytes from the top of the stack.

It is important to note that the fourth parameter or new memory offset to write should be divisible by 4. This is to prevent illegal bytes.

(call $fdw
 (i32.const 1)
 (i32.const 0)
 (i32.const 1)
 (i32.const 20)
)
Enter fullscreen mode Exit fullscreen mode

Note: Having the fourth parameter i32.const 18 will result in !!! bad alignment: 18 % 4.

If you want to get the complete trace of what is happening, which function is called and how much time is spent on each function call. We can easily track that down using the -d flag.

$ RUST_LOG=trace wasmtime writer.wat -d
Enter fullscreen mode Exit fullscreen mode

The above code will print the trace level output of the logs. It will also print all the code transformation that is done by the wasmtime.

TRACE wasmtime_wasi_c::syscalls > fd_write(fd=1, iovs=0x0, iovs_len=1, nwritten=0xc)
Hooray it's WASI
 TRACE wasmtime_wasi_c::syscalls > | *nwritten=20
 TRACE wasmtime_wasi_c::syscalls > -> errno=__WASI_ESUCCESS
Enter fullscreen mode Exit fullscreen mode

This is the trace output from the actual WASI system calls.

Also if you find it difficult to use the syscall at any point because you cannot determine the type of the syscall function. It is better to debug what is failing with tools like wabt (wat2wasm executable) in general and then alter the function definition as needed.

Errors to note in WASI

There are various errors that WASI emits that will help you while developing we will see a few of them here. Every syscall in WASI (at least for now) is blocking and not asynchronous. This means that once you call the syscall, you have to wait till the call is completed and proceed further based on the result.

Getting Started

We will start with an example. In this example, we will create a new directory using WASI. It is always exciting to write it in WebAssembly Text Format and then using wasmtime to run the WebAssembly Text Format.

Let us create a file called creator.wast.

Open up the editor and we will start writing the WebAssembly Text Format.

How to do it

We will first create a module. All the code will live inside the module.

(module )
Enter fullscreen mode Exit fullscreen mode

Then we import the API for creating the new directory. The function signature of the create directory syscall has three arguments.

__wasi_fd_t -> The file descriptor
const char *ptr -> This should be a pointer to the start of the linear memory array inside the WebAssembly Module
size_t path_len -> The length of the value that is stored in the memory array.

The imported function will look like this

(module
 (import "wasi_unstable" "path_create_directory" (func $mkdir (param i32 i32 i32) (result i32))
)
Enter fullscreen mode Exit fullscreen mode

We will define the memory, like this.

(module
 ;; define the imported function
 (memory 1)
 (export "memory" (memory 0))

 (data (i32.const 12) "wasi-folder")
)
Enter fullscreen mode Exit fullscreen mode

We are defining the data in the linear memory array at an offset 12.

Now we have to define the start method that will call the $mkdir function.

(module
 ;; define the imported function
 ;; define memory and data

 (func $main (export "_start")
 (i32.store (i32.const 0) (i32.const 12))
 (i32.store (i32.const 4) (i32.const 20))

 (call $mkdir
 (i32.const 3)
 (i32.const 12)
 (i32.const 11)
 )
 drop
 )
)
Enter fullscreen mode Exit fullscreen mode

How it works

The $mkdir takes in three parameters.

  1. The first one being the file descriptor. We have passed in an i32.const 3 that takes in the third value in the process argument. (the --dir argument)
  2. The next is the starting of the data defined. The data starts at the offset 12. So we will use the same offset here.
  3. Finally the length of the data that we have. In our case, it is 11.

Once done, let us run our application using wasmtime.

$ /path/to/wasmtime/target/wasmtime creator.wat --dir=.
$ ls | grep wasi-folder
drwxr-xr-x 2 sendilkumar staff 64B Jun 12 11:35 wasi-folder
Enter fullscreen mode Exit fullscreen mode

Note that we have given the current directory using the notation "." but notations like ".." will work with the wasmtime if provided to the --dir option.

But if we change the data into (data (i32.const 12) "../some_folder"

And then extend the memory offset to reflect the changes, will result in a __WASI_ENOTCAPABLE error.

The not capable error tells that the WASI is trying to create or access something that it did not have access.

If we pass in zero bytes as the length of the path, then the WASI will throw __WASI_ENOENT error.

The no ent error tells that there is no file or directory available.

__WASI_EILSEQ error is thrown when there is any illegal sequence of the data from linear memory is accessed.

If you have enjoyed the post, then you might like my book on Rust and WebAssembly. Check them out here

Further Explorations

Check out this awesome post from Lin Clark about WASI or WebAssembly System Interface here

Read more about how to use wasmtime with C or C++ here

Check out more available APIs here

Check out more about the WASI tutorial here

Check out for more errors in the WASI_API here

Check out more about the WASI Background here

Discussions 🐦 Twitter // 💻 GitHub // ✍️ Blog

If you like this article, please leave a like or a comment. ❤️

Top comments (4)

Collapse
 
wolfchamane profile image
Arturo Martínez Díaz •

So, you write code that already runs natively (previous compilation process) to run it throught a kind of virtual machine so it can run natively. Nothing more to say.

Collapse
 
ytjchan profile image
ytjchan •

It has some cross-platform quality to it, like you can run the same C++ program in Windows and iOS without recompiling it (like in Java times). Not sure about performance though.

Collapse
 
wolfchamane profile image
Arturo Martínez Díaz •

IMO the main target to develop code that must be compiled to run is to improve performance. Java, ES6+ or any other uncompiled code don't reaches great terms of peformance against compiled ones. The target of uncompiled is "develop one, run everywhere".

To sum up: taking a code which main target is perfomance, transpile it to a cross-platform version (loosing performance throug the process) and then execute it everywhere using a VM ... it sounds, kind of ankward for me.

Thread Thread
 
sendilkumarn profile image
Sendil Kumar •

So, you write code that already runs natively

Thats a very good question. The biggest advantage is portability. Think in this way you need a language specific runtime to run your code. What if you have a single runtime that runs all your code (irrespective of multiple languages).

Java, ES6+ or any other uncompiled code don't reaches great terms of performance

Do you mean JavaScript here? Yeah with WASI you have a runtime the gives you a better performance than JS (NodeJS) alternative.

You will not use that level of performance. But yes, if you just want to write C++, WASI is not an option.

Read next

shilleh profile image

How to Use L298N Motor Driver with Raspberry Pi Pico W

Shilleh -

furkangozukara profile image

Thoroughly experimented with Fine-Tuning / DreamBooth training of Flux-dev-de-distill, PixelWave v03, Verus Vision

Furkan Gözükara -

angesh_singh profile image

How To Use Housing App Full Tutorial

Angesh Singh -

emmanuelayinde profile image

Understanding merge sort algorithm: Beginner's guide to mastering sorting algorithm

Emmanuel Ayinde -