The Watermelon Operator

Structured concurrency
1
1 Like
2

Ended up writing a post that explains Rust’s “natural” structured concurrency approach (if you don’t spawn tasks, and just join futures, you are structured by construction), looks at its boundaries and hypotheses about how far further it can be pushed!

3

To be honest I found this hard to follow. Is this a fair summary?

First there’s discussion of the posts by withoutboats :

  • withoutboats introduces “multi-task” and “intra-task” concurrency - but you’re not happy with this distinction
  • your examples of these:
    • spawn update_db() and update_cache() and returns without waiting (“multi-task” style)
    • join(update_db(), update_cache()) (“intra-task” style)
  • The rest of the post seems to be about the second type, which uses join()
  • It seems like you’re going to revisit this at the end of the post, perhaps to suggest an alternative way of dividing up types of concurrency, or why there is no division at all, but you never do.

Then you talk about join() and structured concurrency:

  • In Rust you can join a list of coroutines (join(a(), b()).await), which acts a bit like a nursery
  • But there’s a difference: join() can only accept a fixed-length list of routines, whereas a nursery lets you arbitrarily spawn tasks into it at runtime
  • For example, if you have a nursery that listens for incoming connections, and spawns a task for each connection received, then you could need arbitrarily many over time.
  • A possible resolution to this, rather than updating join() to handle extra tasks over time, is to instead to use a new primitive, which this post calls the watermelon operator or concurrently()
  • Between them, the new primitive you’re suggesting and the existing join() function gives the same overall functionality of nuseries.
  • There’s a discussion about memory allocation, suggesting that a second stack for the thread would do for all async tasks (surely not if this can spawn an arbitrary tree of tasks).
  • Finally, there’s discussion about cancellation, where you have to manually create and pass around cancellation tokens (like C#)

But I still don’t understand what concurrently() / the watermelon operator actually does. why does it take a condition? It certainly seems far more complicated that asking a nursery to start a new task.

Also, although I do see this adds one feature from nurseries, I don’t think it handles all the features / guarantees:

  • I’m not clear on whether memory is reclaimed if tasks finish in arbtrary order (maybe it is - I didn’t follow the memory part). E.g. if your nursery-like pattern spawns a new child task (1), but that stays alive for a very long time, while tasks (2), (3), … all start and stop fairly quickly, does the memory for later tasks stay active until (1) finally finishes?
  • It doesn’t stop you from spawn tasks in some other way (I think), whereas if you await a Trio function (and don’t pass it a nursery) then you know that no new tasks are running in the background when it finishes.
  • It doesn’t allow passing nurseries around to bypass no-spawn restriction in a controlled way e.g. in Python I can write a function start_something(nursery, …) and it can spawn a task that continues after the function returns, but I still have control over when the task finishes (by choosing which nursery to pass it).
  • It doesn’t automatically cancel within a scope - you have to manually pass and track cancellation tokens. It’s like you’re still only halfway down the path to discovery that would lead you to cancellation scopes (look at NJS’s “cancellation for humans” post from 2019, which talks about cancellation tokens on the way to cancellation scopes).
  • A key feature of nurseries is that when one task returns an error, the others are cancelled and we wait for cancellation to complete. But I don’t think async Rust join() does that at the moment (in fact I don’t think Rust tasks can do work in response to cancellation at all - they just get dropped without notice) and this doesn’t seem to propose changing that.

I’d argue that nurseries are still preferable to your new suggested primitive - much simpler to understand, safer and more powerful. But I’m still not sure I’ve fully understood.