Contexts and capabilities initiative

initiative status: active

What is this?

This page tracks the work of the Contexts and capabilities initiative! To learn more about what we are trying to do, and to find out the people who are doing it, take a look at the charter.

Current status

The following table lists of the stages of an initiative, along with links to the artifacts that will be produced during that stage.

StageStateArtifact(s)
Proposal✅Proposal issue
Charter
Tracking issue
Experimental🦀Evaluation
RFC
Development💤Explainer
Feature complete💤Stabilization report
Stabilized💤

Key:

  • ✅ -- phase complete
  • 🦀 -- phase in progress
  • 💤 -- phase not started yet

How Can I Get Involved?

  • Check for 'help wanted' issues on this repository!
  • If you would like to help with development, please contact the owner to find out if there are things that need doing.
  • If you would like to help with the design, check the list of active design questions first.
  • If you have questions about the design, you can file an issue, but be sure to check the FAQ or the design-questions first to see if there is already something that covers your topic.
  • If you are using the feature and would like to provide feedback about your experiences, please [open a "experience report" issue].
  • If you are using the feature and would like to report a bug, please open a regular issue.

We also participate on {{CHAT_PLATFORM}}, feel free to introduce yourself over there and ask us any questions you have.

Building Documentation

This repository is also an mdbook project. You can view and build it using the following command.

mdbook serve

✏️ Updates

Lang-team initiatives give monthly updates. This section collects the updates from this initiative for posterity.

To add a new update:

  • Create a new file updates/YYYY-mmm.md, e.g. updates/2021-nov.md
  • Link it from the SUMMARY.md

📜 Contexts and capabilities Charter

Context objects in Rust must be passed manually today. This means that they are difficult or impossible to use with code you don't control. It also pushes programmers toward making context objects more coarse-grained than they have to be and using globals where they aren't desirable.

The goal of this initiative is to create a first-class way of working with contexts in Rust that solves these problems:

  • It should be possible to thread contexts through code you don't control, without the code having to make any accommodations.
  • Contexts are allowed to exist on the stack.
  • Contexts can be generic / type erased.
  • Contexts can be "decomposed" into smaller pieces or "bundled" into larger ones.
  • Contexts can be used to implement compile-time dependency injection.
  • Contexts work well with dyn, higher order functions, and the like.

Eventually this mechanism can be used to solve a number of other challenges in the language and standard library:

  • Making "global" objects like allocators and async executors defineable and overridable by the user.
  • Describing one's runtime environment in a more detailed way than core/alloc/std, which is not always appropriate for runtimes like wasm.
  • Making the Rust ecosystem more composable and interoperable.

Proposal

TODO

Membership

RoleGithub
Ownerghost
Liaisonghost

🔬 Evaluation

The evaluation surveys the various design approaches that are under consideration. It is not required for all initiatives, only those that begin with a problem statement but without a clear picture of the best solution. Often the evaluation will refer to topics in the design-discussions for more detailed consideration.

Syntax and naming

This page describes various keywords and surface-level syntactical approaches that can be used to implement the feature. More may be added as new ideas arise.

Defining context items

Keyword

The original blog post about this feature called context items a capability:


#![allow(unused)]
fn main() {
capability basic_arena<'a> = &'a BasicArena;
}

Some people have pushed back against this. While these items are related to capabilities, that isn't the full story. A more neutral name is context.


#![allow(unused)]
fn main() {
context basic_arena<'a> = &'a BasicArena;
}

Types and bounds

Context objects are simply items that may have where clauses attached to them. This means it is possible to define a "bare" context with no type:


#![allow(unused)]
fn main() {
context anything_goes;
}

Or a specific type, or a generic context with bounds:


#![allow(unused)]
fn main() {
context basic_arena = BasicArena;
context any_allocator: Allocator;
context debug_iter: Iterator where Self::Item: Sized;
}

It's unclear what the best way is to write these. People might be confused by the use of = to name an exact type. One approach is to treat them just like function arguments.


#![allow(unused)]
fn main() {
context basic_arena: BasicArena;
context any_allocator: impl Allocator;
}

But then we also want to be able to write where clauses on the type, which you can do with argument-position impl Trait. How would we write debug_iter?


#![allow(unused)]
fn main() {
// This could work...
context debug_iter: impl Iterator where debug_iter::Item: Sized;

// ...or we could allow you to write this.
context debug_iter: impl Iterator where Self::Item: Sized;
}

The problem with using Self is that it looks more like could we would write in a trait definition, where : is used to denote supertraits. It's not clear whether this would be confusing or not, just something to consider.

Whatever we choose, we would probably want given clauses to act the same way.

Note that in where clauses we treat the name of the context as a type (as in the first example above), while in expression position we treat it as a value.

Providing and demanding context

The original blog post about this feature introduced the with keyword for both introducing contexts in an expression and requiring them as clauses.

fn deserialize<'a>(bytes: &[u8]) -> Result<&'a Foo, Error>
with
    arena::basic_arena,
{
    arena::basic_arena.alloc(Foo::from_bytes(bytes)?)
}

fn main() -> Result<(), Error> {
    with arena::basic_arena = &arena::BasicArena::new() {
        let foo: &Foo = deserialize(read_bytes()?)?;
        println!("foo: {:?}", foo);
    }
    Ok(())
}

However, it was pointed out that with can mean many things, and more specific keywords could be used like given:

fn deserialize<'a>(bytes: &[u8]) -> Result<&'a Foo, Error>
given
    arena::basic_arena,
{
    arena::basic_arena.alloc(Foo::from_bytes(bytes)?)
}

fn main() -> Result<(), Error> {
    given arena::basic_arena = &arena::BasicArena::new() {
        let foo: &Foo = deserialize(read_bytes()?)?;
        println!("foo: {:?}", foo);
    }
    Ok(())
}

This can still be used in both places. It's arguably much more clear than with clauses that a given clause requires context to be provided to it.

Moves and mutation

Should we allow passing contexts (including non-Copy contexts) by value, or require them to be passed by reference?

Option 1: Shared references only

This is the simplest option; shared references are always Copy. We can save users the trouble of writing & everywhere by implicitly adding it for them. Contexts that need mutation can trivially add it with RefCell or Mutex.

Option 2: Shared and mutable references only

Here we would always know that a declared context object remains available, but not whether we could hold a valid reference to it. I'm not sure why it would be any easier to implement than option 3.

Option 3: Allow passing by value or by reference

This option is attractive because it means contexts are not special; they behave just like other arguments. However, it might be significantly less ergonomic. It could also have unexpected consequences we haven't thought of.

This blog post has thoughts on how this could work. It raises the idea of treating contexts like closure captures in that they are (mutability-inferred) references by default, but can be explicitly declared move. Thinking of contexts as captures rather than regular arguments in this sense could be justification for such an idea.

Captures and overrides

Consider:

impl Deserialize for Bar
given
    basic_arena: BasicArena,
{ ... }

fn deserialize_and_print<T>(input: impl Read) -> T
where
    T: Deserialize + Debug,
{ ... }

fn main() {
    given basic_arena = BasicArena::new() {
        // This works, even though `deserialize_and_print` knows nothing about
        // `basic_arena`!
        let _foo: &Foo = deserialize_and_print(std::io::stdin());
    }
}

The call to deserialize_and_print above knows that basic_arena has been provided, and therefore that the T: Deserialize bound can be satisfied by the impl.

Mechanically, how do we thread the arena through the impl? Is its value captured at the time of using the impl (meaning in main, at the call to deserialize_and_print) and "smuggled" through somehow? Or do we only capture the knowledge that this context is available, and allow it to be overridden?

More specifically, what happens if we do this?

fn main() {
    given basic_arena = BasicArena::new() {
        let _foo: &Foo = deserialize_and_print_with_arena(std::io::stdin());
    }
}

fn deserialize_and_print_with_arena<T>(input: impl Read) -> T
where
    T: Deserialize + Debug,
{
    // Which arena will win, the caller's or ours?
    given basic_arena = BasicArena::new() {
        deserialize_and_print(input)
    }
}

Note that this is a pretty contrived example. The above function just creates an arena for no particular reason. If we had written its signature just a little differently:


#![allow(unused)]
fn main() {
fn deserialize_and_print_with_arena<T>(input: impl Read) -> T
where
    T: given(basic_arena) Deserialize + Debug,
}

Then it would be obvious what the answer should be: The function "knows" that T: Deserialize requires basic_arena, so its definition should always win. So this ambiguous situation may not happen that often in practice.

It still could, however, if we are dealing with one generic argument or trait object that has already "captured" some context, and another that explicitly requires that context. If the caller provides the context to the latter, should it override the context used by the former?

Using different contexts for each object would surely create confusion for the user. Would this come up often in practice? Is there a way to mitigate the impact somehow: lints, debugging tools, ...?

Continuity of contexts

See also: Captures and overrides

Should there be a way of expressing that the same context is used across multiple calls / interactions with an object?

TODO

This specifically came up when talking about providing a Hasher context for HashMap.

Bundling and splitting

It might be nice to be able to declare a single context that includes many "sub-contexts", then split it up into those parts as needed.

Examples:

  • Full I/O capabilities of a "regular process"
    • Output only
    • Networking
    • Clocks
  • Async executor
    • Task spawner
    • Timer management

Syntax

If we use the more argument-like syntax to define context types, we could reclaim = for this:


#![allow(unused)]
fn main() {
context foo;
context bar;
context baz;

context kaboodle = foo + bar + baz;
}

Migrating existing APIs

If a library with an existing API could benefit from using context objects in its public API, is there a soft migration path that doesn't require both the library and its users to change at the same time?

Injectable arguments

An earlier proposal for #[inject]able functions hints at an answer: make contexts regular function arguments, and allow callers to leave off arguments that are already given.


#![allow(unused)]
fn main() {
fn do_stuff(
    num: u32,
    #[inject(given log::logger)]
    logger: Logger,
) {
    info!(logger, "do_stuff({})", num);
}
}

This is similar to how implicit arguments work in Scala. There are some challenges, however.

  • What happens if some of your #[inject] arguments are provided and some aren't?
  • Should the function signature effectively be rewritten to remove the provided arguments in a given scope? Spooky!
  • It's tempting to only allow #[inject] arguments at the end to reduce confusion. However, it would severely limit its usefulness for migrating existing APIs.

📚 Explainer

The "explainer" is "end-user readable" documentation that explains how to use the feature being deveoped by this initiative. If you want to experiment with the feature, you've come to the right place. Until the feature enters "feature complete" form, the explainer should be considered a work-in-progress.

✨ RFC

The RFC exists here in draft form. It will be edited and amended over the course of this initiative. Note that some initiatives produce multiple RFCs.

Until there is an accepted RFC, any feature gates must be labeled as experimental.

When you're ready to start drafting, copy in the template text from the rfcs repository.

😕 Frequently asked questions

This page lists frequently asked questions about the design. It often redirects to the other pages on the site.

What is the goal of this initiative?

See the Charter.

Who is working on it?

See the Charter.

How do other languages approach similar problems?