Squared: Write once, Rust anywhere

This is a README FROM THE FUTURE, in that it described the workflow for something that doesn't exist yet.

Squared is a project for authoring pure-Rust libraries that can be integrated into any language on any operating system or environment. Squared can help you...

  • publish a cross-language API for accessing your cloud service;
    • squared currently supports Java/Kotlin, Python, JavaScript, C, C++, and Go, but adding new languages is easy.
  • package up common code for use across mobile devices.

Want to see how easy squared can be? Check out our tutorial.

Pick your poison

squared can be used in three modes:

ModePerformanceSandboxingDistribution
Raw FFI😎 Native⚠️ NoPortable binary artifact
Sandboxed FFI using rlbox.devGood✅ Yes!Portable binary artifact
WebAssembly componentGood✅ Yes!WASM module running anywhere, including the web

Here are the key differences between the modes

  • Raw FFI -- compiles Rust to native code and packages the library as a binary artifact with cosmopolitan. This offers the best performance, particularly for libraries that make good use of SIMD, but (a) means that you have to distribute a binary artifact, which can be a hassle; and (b) does not offer allow the library to be sandboxed.
  • Sandboxed FFI -- compiles Rust to WebAssembly and then uses rlbox.dev to compile that to native code. Indirecting through WebAssembly costs some performance (typically around 10%) but gives the benefit of sandboxing. This means that the Rust code can be treated as untrusted by the host application.
  • WebAssembly component -- compiles Rust to WebAssembly. This comes with a slight performance hit but offers sandboxing and means that you can distribute one binary that runs everywhere (including in browsers!).

Tutorial

How squared works

You start by creating a Rust library whose public interfaces follows the squared conventions, which means that you stick to Rust types and features that can readily be translated across languages. The body of those functions can make use of whatever logic you want. For example, suppose you wanted to publish some logic based on Rust's best-in-class [regex][] library. You might write:

#![allow(unused)]
fn main() {
pub fn find_username(s: &str) -> String {
    let r = regex::compile("@([a-zA-Z]+)").unwrap();
    if let Some(m) = r.captures(s) {
        m.to_string()
    } else {
        panic!("no username found")
    }
}
}

You would then install and run squared:

> cargo install squared
> cargo squared build

Since you don't have a squared.toml, you'll be asked a few questions, and then squared will run. The result will be a set of libraries that allow your code to be used transparently from other languages. You can also run cargo squared setup if you prefer to just run the setup commands and not do the actual build.

More advanced Rust code

The find_username function is fairly basic. squared supports more advanced interfaces as well.

Public item types

squared works by parsing your lib.rs module to determine your public interface. It only allows the following kinds of pub items:

  • pub fn to define a public function.
  • pub struct or pub enum to define a public struct, enum, or class (see below).
  • pub use crate::some::path to publish some part of your crate.

You will get an error if you have other public items in your lib.rs because squared does not know how to translate them to a public API. If you wish to include them anyway, you can tag them with the #[squared::ignore] attribute. This will cause them to be ignored, which means that they will only be available to Rust consumers of your library.

Basic Rust types

You can use the following built-in Rust types in your public interfaces:

  • numeric scalar types like i8, u16, f32 up to 64 bits;
  • char;
  • &str and String;
  • Slices (&[T]) and vectors (Vec<T>), where T is some other supported type;
  • Maps (HashMap, BTreeMap, IndexMap) and sets (HashSet, BTreeSet, IndexSet);
  • Options Option<T> and results Result<T, U>;
  • tuples.

Function parameters can also be &-references to the above types, e.g., &HashSet<String> (in fact, this is recommended unless ownership is truly required).

Simple structs and enums

You can define public structs and enums:

#![allow(unused)]
fn main() {
/// Translated to a WebAssembly [record][]
/// 
/// [record]: https://component-model.bytecodealliance.org/design/wit.html#records
pub struct MyStruct {
    pub field: T,
}

/// Enums with no values are translated to a WebAssembly enum,
/// which means they will be represented in target languages as
/// the native enum construct.
pub enum MySimpleEnum {
    Variant1,
    Variant2,
}

/// Enums with no values are translated to a WebAssembly enum,
/// which means they will be represented in target languages as
/// the native variant construct.
pub enum MyComplexEnum {
    Variant1(T),
    Variant2,
}
}

"Classes" (types with methods)

#![allow(unused)]
fn main() {
/// Translated to a WebAssembly [resource][]
/// 
/// [record]: https://component-model.bytecodealliance.org/design/wit.html#records
pub struct MyResource {
    field: T,
}

impl MyResource {
    pub fn new() -> Self {

    }

    pub fn method1(&self) {

    }

    pub fn static_method1(&self) {

    }
}
}

WebAssembly

Configuration

Frequently asked questions

Why the name squared?

The name squared comes from the idea that this package enables clean interop between various languages. Ordinarily that would require N^2 different bits of code, but since squared leverages WebAssembly's interface types, we can enable interop with just one.

Reference

Defining your public interface

squared works by parsing your lib.rs module to determine your public interface. It only allows the following kinds of pub items:

  • pub fn to define a public function.
  • pub struct or pub enum to define a public struct, enum, or class (see below).
  • pub use crate::some::path to publish some part of your crate.

Public functions

You can declare top-level Rust functions:

#![allow(unused)]
fn main() {
pub fn greet(name: &str) -> String {
    format!("Hello, {name}!")
}
}

The argument and return types of these functions have to consist of translatable Rust types.

Structs defined with the "class" pattern

Squared recognizes the common Rust idiom of a public struct with private members and public methods defined in an impl block. This pattern is called the class pattern and, for OO languages, it will be translated into a class.

#![allow(unused)]
fn main() {
pub struct MyClass {
    // Fields must be private
    field1: Field1
}

impl MyClass {
    /// If you define a `new` function, it becomes the constructor.
    /// Classes can have at most one constructor.
    pub fn new() -> Self {}

    /// Classes can only have `&self` methods.
    pub fn method(&self) {}

    /// Classes can also have "static" methods with no `self`.
    pub fn static_method() {}
}
}

Public structs and enums

You can define public structs and enums. The contents of these types must be fully public, which also means you are committed to not changing them.

#![allow(unused)]
fn main() {
/// Translated to a WebAssembly [record][]
/// 
/// [record]: https://component-model.bytecodealliance.org/design/wit.html#records
pub struct MyStruct {
    pub field: T,
}

/// Enums with no values are translated to a WebAssembly enum,
/// which means they will be represented in target languages as
/// the native enum construct.
pub enum MySimpleEnum {
    Variant1,
    Variant2,
}

/// Enums with no values are translated to a WebAssembly enum,
/// which means they will be represented in target languages as
/// the native variant construct.
pub enum MyComplexEnum {
    Variant1(T),
    Variant2,
}
}

Public uses

You include a pub use to import things from elsewhere in your crate and include them in your public interface. You must write the use in absolute form:

#![allow(unused)]
fn main() {
pub use crate::path::to::Something;
}

squared will look for the definition of Something in src/path/to.rs.

Private members and ignored items

Normally all public entries defined in your lib.rs must be fit one of the above categories so that squared knows how to translate them. You can also have arbitrary Rust code so long as the items are private to your crate.

Sometimes you would like to include public Rust members that are not part of your public interface. You can do that by annotation those members with #[squared::ignore].

Translating Rust types

Your public functions and methods can use the following Rust types.

  • numeric scalar types like i8, u16, f32 up to 64 bits;
  • char;
  • &str and String;
  • tuples, options Option<T> and results Result<T, U>;
  • collection types:
    • slices (&[T]) and vectors (Vec<T>)
    • maps (HashMap, BTreeMap, IndexMap)
    • sets (HashSet, BTreeSet, IndexSet)
  • user-defined types in your library:
  • user-defined types from other squared libraries:
    • XXX importing from other libraries?

Function parameters can be &-references to the above types.

Function return types must be owned.

Toll-free bridging

Using native Rust types for collections is convenient but can incur a performance cost as data must be copied out from native collections into the Rust type and vice versa. To avoid this you can use "toll-free" bridging in your Rust code: this means that you code traits defined in the squared stdlib:

  • impl MapLike<K,V>
  • impl VecLike<T>
  • impl SetLike<E>

You can also write your function to return a type R implementing one of those traits. Squared will recognize this pattern and pick appropriate instances of those traits for best performance. For example, for C++, MapLike can be instantiated with the STL maps, avoiding the need to copy data into a Rust map. In some cases multiple variants may be created (e.g., if the function is invoked multiple times).

The Squared IDL

The squared IDL is available from the squared::idl crate.

Target mappings

WebAssembly Interface Types

The Squared IDL can be directly mapped to WebAssembly interface types, for the most part:

  • Primitive types map directly to WIT primitive types.
  • Collection types map to WIT lists:
    • A Rust Vec<T> or HashSet<T> maps to a WIT list<T>.
    • A Rust HashMap<K, V> maps to a WIT list<tuple<K, V>>.
  • Public structs/enums are mapped to WIT records, variants, and enums as appropriate:
    • A struct is mapped to a WIT record.
    • Enums are mapped to WIT enums when possible, else WIT variants.
  • Instances of the class pattern are mapped to WIT resource types:
    • The methods can be mapped directly.

Mapping to Java

The Squared IDL is mapped to Java as follows:

  • Primitive types:
    • i8, u8 to Java byte
    • i16, u16 to Java short
    • i32, u32 to Java int
    • u64, u64 to Java long
    • f32 to Java float
    • f64 to Java double
    • char to Java u32 (a Java char is not a 32-bit unicode code point)
  • Collection types map to Java collections:
    • A Rust Vec<T> to a Java ArrayList<T>
    • ...
  • Tuples and public structs map to Java classes with public fields
  • Enums with associated data map to an abstract Java base class and public-struct-like subclasses for each variant
  • Enums map without associated data map to Java enums
  • Instances of the class pattern map to Java classes with methods

Mapping to C

The Squared IDL is mapped to Java as follows:

  • Primitive types map to C in the obvious ways
  • Enums map without associated data map to Java enums
  • For all other types, we generate C wrappers of various kinds, as described below

Collections

We will create new struct types for each collection as needed.

These will include helper methods to:

  • create an empty collection and insert new elements;
  • read the "length" of the collection;
  • iterate over the collection;
  • for vectors, access the ith member or treat the collection as a C array;
  • for sets, access the ith member or treat the collection as a C array;
  • for maps, lookup the element for a key.

Mapping to C++

Similar to C but can make use of the C++ STL.

API descriptions in squared

  • API descriptions are
    • exported structs, enums
    • exported classes (struct + impl)
    • exported public functions