Squared: Write once, Rust anywhere
This is a README FROM THE FUTURE, in that it described the workflow for something that doesn't exist yet.
Squared is a project for authoring pure-Rust libraries that can be integrated into any language on any operating system or environment. Squared can help you...
- publish a cross-language API for accessing your cloud service;
- squared currently supports Java/Kotlin, Python, JavaScript, C, C++, and Go, but adding new languages is easy.
- package up common code for use across mobile devices.
Want to see how easy squared can be? Check out our tutorial.
Pick your poison
squared can be used in three modes:
Mode | Performance | Sandboxing | Distribution |
---|---|---|---|
Raw FFI | 😎 Native | ⚠️ No | Portable binary artifact |
Sandboxed FFI using rlbox.dev | Good | ✅ Yes! | Portable binary artifact |
WebAssembly component | Good | ✅ Yes! | WASM module running anywhere, including the web |
Here are the key differences between the modes
- Raw FFI -- compiles Rust to native code and packages the library as a binary artifact with cosmopolitan. This offers the best performance, particularly for libraries that make good use of SIMD, but (a) means that you have to distribute a binary artifact, which can be a hassle; and (b) does not offer allow the library to be sandboxed.
- Sandboxed FFI -- compiles Rust to WebAssembly and then uses rlbox.dev to compile that to native code. Indirecting through WebAssembly costs some performance (typically around 10%) but gives the benefit of sandboxing. This means that the Rust code can be treated as untrusted by the host application.
- WebAssembly component -- compiles Rust to WebAssembly. This comes with a slight performance hit but offers sandboxing and means that you can distribute one binary that runs everywhere (including in browsers!).
Tutorial
How squared works
You start by creating a Rust library whose public interfaces follows the squared conventions, which means that you stick to Rust types and features that can readily be translated across languages. The body of those functions can make use of whatever logic you want. For example, suppose you wanted to publish some logic based on Rust's best-in-class [regex][] library. You might write:
#![allow(unused)] fn main() { pub fn find_username(s: &str) -> String { let r = regex::compile("@([a-zA-Z]+)").unwrap(); if let Some(m) = r.captures(s) { m.to_string() } else { panic!("no username found") } } }
You would then install and run squared
:
> cargo install squared
> cargo squared build
Since you don't have a squared.toml
, you'll be asked a few questions, and then squared will run. The result will be a set of libraries that allow your code to be used transparently from other languages. You can also run cargo squared setup
if you prefer to just run the setup commands and not do the actual build.
More advanced Rust code
The find_username
function is fairly basic. squared
supports more advanced interfaces as well.
Public item types
squared works by parsing your lib.rs
module to determine your public interface. It only allows the following kinds of pub
items:
pub fn
to define a public function.pub struct
orpub enum
to define a public struct, enum, or class (see below).pub use crate::some::path
to publish some part of your crate.
You will get an error if you have other public items in your lib.rs
because squared does not know how to translate them to a public API. If you wish to include them anyway, you can tag them with the #[squared::ignore]
attribute. This will cause them to be ignored, which means that they will only be available to Rust consumers of your library.
Basic Rust types
You can use the following built-in Rust types in your public interfaces:
- numeric scalar types like
i8
,u16
,f32
up to 64 bits; char
;&str
andString
;- Slices (
&[T]
) and vectors (Vec<T>
), whereT
is some other supported type; - Maps (
HashMap
,BTreeMap
,IndexMap
) and sets (HashSet
,BTreeSet
,IndexSet
); - Options
Option<T>
and resultsResult<T, U>
; - tuples.
Function parameters can also be &
-references to the above types, e.g., &HashSet<String>
(in fact, this is recommended unless ownership is truly required).
Simple structs and enums
You can define public structs and enums:
#![allow(unused)] fn main() { /// Translated to a WebAssembly [record][] /// /// [record]: https://component-model.bytecodealliance.org/design/wit.html#records pub struct MyStruct { pub field: T, } /// Enums with no values are translated to a WebAssembly enum, /// which means they will be represented in target languages as /// the native enum construct. pub enum MySimpleEnum { Variant1, Variant2, } /// Enums with no values are translated to a WebAssembly enum, /// which means they will be represented in target languages as /// the native variant construct. pub enum MyComplexEnum { Variant1(T), Variant2, } }
"Classes" (types with methods)
#![allow(unused)] fn main() { /// Translated to a WebAssembly [resource][] /// /// [record]: https://component-model.bytecodealliance.org/design/wit.html#records pub struct MyResource { field: T, } impl MyResource { pub fn new() -> Self { } pub fn method1(&self) { } pub fn static_method1(&self) { } } }
WebAssembly
Configuration
Frequently asked questions
Why the name squared?
The name squared comes from the idea that this package enables clean interop between various languages. Ordinarily that would require N^2 different bits of code, but since squared leverages WebAssembly's interface types, we can enable interop with just one.
Reference
Defining your public interface
squared works by parsing your lib.rs
module to determine your public interface. It only allows the following kinds of pub
items:
pub fn
to define a public function.pub struct
orpub enum
to define a public struct, enum, or class (see below).pub use crate::some::path
to publish some part of your crate.
Public functions
You can declare top-level Rust functions:
#![allow(unused)] fn main() { pub fn greet(name: &str) -> String { format!("Hello, {name}!") } }
The argument and return types of these functions have to consist of translatable Rust types.
Structs defined with the "class" pattern
Squared recognizes the common Rust idiom of a public struct with private members and public methods defined in an impl
block. This pattern is called the class pattern and, for OO languages, it will be translated into a class.
#![allow(unused)] fn main() { pub struct MyClass { // Fields must be private field1: Field1 } impl MyClass { /// If you define a `new` function, it becomes the constructor. /// Classes can have at most one constructor. pub fn new() -> Self {} /// Classes can only have `&self` methods. pub fn method(&self) {} /// Classes can also have "static" methods with no `self`. pub fn static_method() {} } }
Public structs and enums
You can define public structs and enums. The contents of these types must be fully public, which also means you are committed to not changing them.
#![allow(unused)] fn main() { /// Translated to a WebAssembly [record][] /// /// [record]: https://component-model.bytecodealliance.org/design/wit.html#records pub struct MyStruct { pub field: T, } /// Enums with no values are translated to a WebAssembly enum, /// which means they will be represented in target languages as /// the native enum construct. pub enum MySimpleEnum { Variant1, Variant2, } /// Enums with no values are translated to a WebAssembly enum, /// which means they will be represented in target languages as /// the native variant construct. pub enum MyComplexEnum { Variant1(T), Variant2, } }
Public uses
You include a pub use
to import things from elsewhere in your crate and include them in your public interface. You must write the use
in absolute form:
#![allow(unused)] fn main() { pub use crate::path::to::Something; }
squared will look for the definition of Something
in src/path/to.rs
.
Private members and ignored items
Normally all public entries defined in your lib.rs must be fit one of the above categories so that squared knows how to translate them. You can also have arbitrary Rust code so long as the items are private to your crate.
Sometimes you would like to include public Rust members that are not part of your public interface.
You can do that by annotation those members with #[squared::ignore]
.
Translating Rust types
Your public functions and methods can use the following Rust types.
- numeric scalar types like
i8
,u16
,f32
up to 64 bits; char
;&str
andString
;- tuples, options
Option<T>
and resultsResult<T, U>
; - collection types:
- slices (
&[T]
) and vectors (Vec<T>
) - maps (
HashMap
,BTreeMap
,IndexMap
) - sets (
HashSet
,BTreeSet
,IndexSet
)
- slices (
- user-defined types in your library:
- simple structs and enums
- structs following the class pattern
- user-defined types from other squared libraries:
- XXX importing from other libraries?
Function parameters can be &
-references to the above types.
Function return types must be owned.
Toll-free bridging
Using native Rust types for collections is convenient but can incur a performance cost as data must be copied out from native collections into the Rust type and vice versa. To avoid this you can use "toll-free" bridging in your Rust code: this means that you code traits defined in the squared stdlib:
impl MapLike<K,V>
impl VecLike<T>
impl SetLike<E>
You can also write your function to return a type R
implementing one of those traits. Squared will recognize this pattern and pick appropriate instances of those traits for best performance. For example, for C++, MapLike
can be instantiated with the STL maps, avoiding the need to copy data into a Rust map. In some cases multiple variants may be created (e.g., if the function is invoked multiple times).
The Squared IDL
The squared IDL is available from the squared::idl
crate.
Target mappings
WebAssembly Interface Types
The Squared IDL can be directly mapped to WebAssembly interface types, for the most part:
- Primitive types map directly to WIT primitive types.
- Collection types map to WIT lists:
- A Rust
Vec<T>
orHashSet<T>
maps to a WITlist<T>
. - A Rust
HashMap<K, V>
maps to a WITlist<tuple<K, V>>
.
- A Rust
- Public structs/enums are mapped to WIT records, variants, and enums as appropriate:
- A struct is mapped to a WIT record.
- Enums are mapped to WIT enums when possible, else WIT variants.
- Instances of the class pattern are mapped to WIT resource types:
- The methods can be mapped directly.
Mapping to Java
The Squared IDL is mapped to Java as follows:
- Primitive types:
i8
,u8
to Javabyte
i16
,u16
to Javashort
i32
,u32
to Javaint
u64
,u64
to Javalong
f32
to Javafloat
f64
to Javadouble
char
to Javau32
(a Javachar
is not a 32-bit unicode code point)
- Collection types map to Java collections:
- A Rust
Vec<T>
to a JavaArrayList<T>
- ...
- A Rust
- Tuples and public structs map to Java classes with public fields
- Enums with associated data map to an abstract Java base class and public-struct-like subclasses for each variant
- Enums map without associated data map to Java enums
- Instances of the class pattern map to Java classes with methods
Mapping to C
The Squared IDL is mapped to Java as follows:
- Primitive types map to C in the obvious ways
- Enums map without associated data map to Java enums
- For all other types, we generate C wrappers of various kinds, as described below
Collections
We will create new struct types for each collection as needed.
These will include helper methods to:
- create an empty collection and insert new elements;
- read the "length" of the collection;
- iterate over the collection;
- for vectors, access the ith member or treat the collection as a C array;
- for sets, access the ith member or treat the collection as a C array;
- for maps, lookup the element for a key.
Mapping to C++
Similar to C but can make use of the C++ STL.
API descriptions in squared
- API descriptions are
- exported structs, enums
- exported classes (struct + impl)
- exported public functions