16. Concurrency
Every task has an owner: a concurrent block or a supervisor. There is no fire-and-forget task. Values are immutable, so tasks share no mutable state, and channels pass immutable values.
concurrent and tasks
Section titled “concurrent and tasks”Taught in Concurrency.
concurrent alone on its line, a body below it and end, is a block expression that opens a task scope. Its value is its last expression. concurrent is an ordinary name anywhere else, so a binding named concurrent read at the end of a line goes in parentheses. Every task started in it has finished or been cancelled before the block’s value is produced.
Control flow in the block is scoped to the block, as in a lambda: return, ?, break and continue leave the block, not the enclosing function. Any exit other than reaching the end cancels every task still running in the block before the exit continues outward.
Task<T>, from std::tasks, is the handle of a running task; T is its body’s result. Task.spawn(body) starts body, a function of no parameters, as a task of the enclosing block, and returns its Task<T> at once. The body runs alongside the code that started it, and the values it captures are shared, immutably. Task.await(task) waits for the task and returns its T. The other task operations (Task.spawn_all, Task.await_all, Task.outcome, Task.cancel) are in std::tasks.
import std::io std::tasks.Task end fn square(n: Int) -> Int n * n end fn main() total = concurrent a = Task.spawn(fn -> square(3)) b = Task.spawn(fn -> square(4)) Task.await(a) + Task.await(b) end io.print("${total}") // 25 end
A failed assert or refute in the block’s own body leaves the block as Err(AssertionFailure), so the block’s value must be a Result<_, AssertionFailure>. Any other value is the error an assertion inside a concurrent block exits the block with Err(AssertionFailure), but the block's value is Int: end the block with a Result<_, AssertionFailure> (`Ok(...)`), or move the assertion out of the block at the assertion. In a test, such a block as the body’s final value fails the case with the assertion’s report.
A task body is a lambda, and an assertion in a lambda is an error (§17 Assertions). To assert in a task, spawn a fn that returns Result<_, AssertionFailure>; its failure is the task’s value, which the code that awaits it handles. A test fails only from its own line of execution.
Static guarantees
Section titled “Static guarantees”The compiler enforces three rules, which together rule out leaked tasks.
Task.spawnruns only inside the dynamic extent of aconcurrentblock. A function that callsTask.spawnmay itself be called from inside a block. A spawn that can run outside every block is the errorspawn outside concurrent block. A task’s body runs in its block’s extent, so a spawn inside it, directly or through a helper at any depth (Task.spawn(fn -> helper())wherehelperspawns), joins that block: the block waits for the task, its failure and cancellation are those of a task spawned in the block’s own body, and cancelling the task that spawned it does not cancel it. Aconcurrentblock inside a task body is a block of its own.- Every
Task<T>is awaited. A task whose handle is never passed toTask.awaitis the errorTask bound to 't' is never awaited. Discarding the result of an await,_ = Task.await(t), counts. Discarding the handle,_ = t, does not, because it skips the wait: it is the error`_ = t` discards the Task without waiting for it; await it instead: `_ = Task.await(t)`. - A
Task<T>does not leave its block. It cannot be returned from the block, sent on a channel, or stored where it outlives the block:Task<T> cannot escape its enclosing concurrent block.
Errors and cancellation
Section titled “Errors and cancellation”Taught in Concurrency.
A task’s error is a value. A fallible body returns Result<T, E>, and the code that awaits it decides what to do:
concurrent a = Task.spawn(fn -> fetch_user(42)) b = Task.spawn(fn -> fetch_order(101)) u = Task.await(a)? // an Err leaves the block and cancels b o = case Task.await(b) Ok(order) -> order Err(_) -> default_order // handled here; nothing is cancelled end Ok((u, o)) end
A task that fails is a different matter: its body trapped, so there is no value to return. Task.await propagates the failure to the code that awaits it, by design, as the trap would have surfaced had the body run inline, and awaiting a task that was cancelled propagates the cancellation. On the main line either ends the program with the task’s fault, as any trap does. Task.await_all propagates a failure the same way, as soon as a task in the block fails. Task.outcome(task) is the total form: it waits as Task.await does and returns how the task ended as a value, Outcome<T> (Failures and restarts below). Code that cancelled a task, or that reports a task’s failure instead of inheriting it, reads the task with Task.outcome.
Cancellation has two sources. Both stop the work at its next safe point (Safe points below), and the stop unwinds the task, running its deferred calls on the way out.
- The block exits early. When a
concurrentblock exits by?,return,breakorcontinue, it cancels every task still running in it. User code does not see this: the cancelled body stops and unwinds. - A deadline or the embedder. Each
concurrentblock derives its context from the application context in force (§15 Context), and its tasks inherit it. When that context’s deadline passes, or the embedder cancels the root, the work under it stops. A task settlesCancelled, and the main line ends with the deadline’s error:deadline exceeded: the context in force when `main` blocked ran out, so the remaining work was not run.
A deadline is set with with App.context = Context.with_timeout(App.context, d) before the work, in the block that holds it. There is no handle that cancels a context. Exceeding the deadline is not a value the block returns: the rest of the region the with line governs is skipped, so code after the concurrent block in that same region cannot report the timeout. To observe it, put the deadline inside a spawned task and read Task.outcome from outside that region.
fn fetch_both() with App.context = Context.with_timeout(App.context, Duration.milliseconds(50)) concurrent a = Task.spawn(fn -> slow_call()) b = Task.spawn(fn -> slow_call()) _ = Task.await(a) _ = Task.await(b) end end
Safe points
Section titled “Safe points”A safe point is every function call, tail calls included; every loop iteration (an Iter.loop step, and each element an iteration passes along, including one with no callback); and every operation that waits (timer.sleep, Sender.send, Receiver.receive, Task.await, a read of standard input). Waiting operations also wait on the context, so cancellation and deadlines reach a task blocked in one.
A task that was cancelled, or whose deadline has passed, stops at the first safe point it reaches, so pure computation stops too. A single call into Go (a host function or a Go binding) is not interrupted; the work stops at the next safe point after it returns. A program must not depend on which safe point a stop lands on.
Channels
Section titled “Channels”Taught in Concurrency.
A channel is a typed FIFO queue between tasks, held as two halves. std::channels declares Channel<T>, a struct of a Sender<T> (ch.sender) and a Receiver<T> (ch.receiver). A function that takes one half can use only that direction; nothing at run time keeps the halves apart.
Sender.sendreturns anErronce the channel is closed, including a send waiting for room when it closes, andReceiver.receivereturnsNoneonce it is closed and drained. Only aSendercloses it, and closing a closed channel does nothing. std::channels has the functions.- A
send,receiveorselectblocked when its context is cancelled unwinds like any cancelled wait. - A channel belongs to no block. It may be made in one block, returned, and used in a later one, and its memory is reclaimed when nothing refers to it.
import std::channels.{Channel, Receiver, Sender} std::io std::tasks.Task end fn produce(out: Sender<Int>, n: Int) -> Unit case Sender.send(out, n) Ok(u) -> u Err(_) -> io.print("send failed") end end fn consume(inbox: Receiver<Int>) -> Int Iter.loop(0, fn sum: Int -> case Receiver.receive(inbox) Some(v) -> sum + v None -> break sum end end) end fn main() -> Unit ch: Channel<Int> = Channel.buffered(4) total = concurrent p1 = Task.spawn(fn -> produce(ch.sender, 10)) p2 = Task.spawn(fn -> produce(ch.sender, 20)) cons = Task.spawn(fn -> consume(ch.receiver)) Task.await(p1) Task.await(p2) Sender.close(ch.sender) Task.await(cons) end io.print("total=${total}") // total=30 end
Waiting on several channels
Section titled “Waiting on several channels”Channel.select(arms) waits until one of a non-empty list of arms can finish, runs that arm’s handler and returns its result. An arm is an Arm<R>: Arm.receive(receiver, handle) passes handle the received Maybe<T>, Arm.send(sender, value, handle) passes it the send’s Result<Unit, ChannelClosed>, and Arm.after(wait, handle) calls it with nothing once wait has passed. The arms may be on channels of different element types; every handler returns the one type R.
import std::channels.{Arm, Channel} std::duration.Duration end fn main() inbox = Channel.buffered<String>(1) reply = Channel.select([ Arm.receive(inbox.receiver, fn message -> Maybe.with_default(message, "closed")), Arm.after(Duration.seconds(0), fn -> "nothing yet"), ]) dbg reply // "nothing yet" end
- The first listed ready arm wins. When several arms can finish as the
selectstarts, the first one in the list finishes and the others take and send nothing, so a run repeats exactly. An arm that is always ready therefore starves the arms listed after it; a loop that serves several channels fairly rotates its arms. While theselectwaits, the first arm to become ready wins. - Closed channels are ready. A receive arm on a closed and drained channel finishes at once with
None, and a send arm on a closed channel withErr(ChannelClosed). So aselectwhose channels are all closed runs its first listed channel arm. - Timers count on the clock in force from the start of the
select: underclock Clock.Virtuala timer fires as soon as every task is waiting, with no real wait. A timer of zero or less is ready at once, soArm.afterof zero listed last makes aselectthat never waits. - The arms are a
NonEmptyList<Arm<R>>, which a list literal of arms is, whatever its arms are (§4 Subtypes); a list built while the program runs goes throughNonEmptyList.try_from. - A waiting
selectis cancellable: when its context is cancelled it unwinds like any cancelled wait, and no handler runs.
Receiver.receive_within(receiver, wait) is the one-channel case: Ok(Some(value)), Ok(None) when the channel is closed and drained, or Err(TimedOut) when wait runs out first. A value already waiting is received even when wait is zero.
Fan-in from producers that share one message type needs no select: they send on one channel of a tagged enum, as std::channels shows. select is for timeouts, sends that must not block, and channels of different types or owners.
Supervision
Section titled “Supervision”Taught in Background Work.
Long-lived background work is owned by a Supervisor, from std::supervisors. A supervisor is a set of tasks that share one policy: a required max_running bound (a PositiveInt), a shutdown budget (shutdown_timeout, with a default for ordinary background work) and a restart policy. max_running is required because no default bound is safe for an unknown downstream.
A supervisor is created only while boot builds the application value, in boot or in something boot calls, and is usually kept in an application field. Anywhere else, Supervisor.new is the error a supervisor here would be a new one every time this runs, and each is kept for the life of the program with a `max_running` of its own — so the bound would be per call rather than per downstream. Create it while `boot` builds the app value: in boot, or in something boot calls.
The checker follows this across the whole program, std included. A function is boot-only when it calls Supervisor.new or another boot-only function, directly or through any chain of calls in any module: std::jobs’s Worker.new is one, and so is an app’s function that calls it. Code starts running outside boot in fn main, in a test body, in a lambda or capture (wherever it is written, since it runs whenever it is called), and in a private function nothing calls. Each call such code makes to a boot-only function is an error at that call, naming the chain down to Supervisor.new: `make_worker` creates a supervisor (through `Worker.new` at workers.nomi:4, then `Supervisor.new` at jobs.nomi:483), so it can only be called while `boot` builds the app value. A call inside a boot-only function is not itself an error; the call into that function is. A pub function nothing calls is not checked from inside: it is another module’s to call. boot, and a function a test group’s boot calls, runs inside boot, and so does an entry boot a test calls. A boot-only function named as a value (f = Worker.new) is an error where it is named, in boot too, since a function value may be called anywhere. The runtime traps with the same message as a backstop.
import std::duration.Duration std::supervisors.Supervisor end struct App context: Context audit: Supervisor end fn boot() -> Result<App, String> Ok(App{ context: Context.root(), audit: Supervisor.new(max_running: 4, shutdown_timeout: Duration.seconds(30)), }) end
Spawning under a supervisor
Section titled “Spawning under a supervisor”Supervisor.spawn(supervisor, body) starts a task under the supervisor and returns a Task<Unit> at once. The body must return Unit, since nobody is required to await it: argument 2: expected () -> Unit, got () -> Int. The handle is the one task handle that may be discarded with _ =; it may also be cancelled, inspected with Task.outcome or awaited.
fn handle_order(id: Int) -> Response response = build_response(id) _ = Supervisor.spawn(App.audit, fn -> record_audit(id, response)) response end
A supervised task may open its own concurrent block; the block owns the tasks it starts, and the supervisor owns the task.
Flushing and shutdown
Section titled “Flushing and shutdown”The program drains every supervisor when main returns or the process receives SIGINT or SIGTERM. Supervisors drain at the same time, each within its own shutdown_timeout; when a budget runs out, the remaining work is cancelled, and shutdown waits up to one second for it to unwind and run its deferred calls before abandoning it. Boot’s deferred calls run after that (§15 Cleanup and boot failure). A second interrupt exits at once.
Failures and restarts
Section titled “Failures and restarts”Task.outcome(task) reports how a task ended, as std::tasks’s Outcome<T>:
enum Outcome<T> Completed T Cancelled Failed Failure end
A body that returns Err(e) produced a value, so its outcome is Completed(Err(e)). Failed is a trap (§11 Traps, Maybe and Result). A supervisor may restart failed work, and never restarts a cancelled task, since cancellation is not a failure. std::supervisors has the restart policies and Supervisor.flush.