Multiple Nurseries For Multiple Steps

Help and advice
1

Asynchronous programming is still new territory for me, so naturally I’m having doubts about my approach to this problem with a REST client.

The problem

To delete an object through the target REST API, other objects associated to the target object must be deleted first. Eg. if

User1
  | Project1
  | Project2

, then I must delete Projects 1 and 2 before I can delete User1.

The goal

I want to pass a list of Users, delete all of their projects, then delete the users.

The approach

async def delete_users_and_projects(users: Sequence[str]) -> Tuple[str, ...]:
    # Have to first get each user's projects. Since it could be an arbitrarily long list,
    # I want to do this step as tasks.
    usr_prj = {}
    async with trio.open_nursery() as n:
        for user in users:
            n.start_soon(get_user_projects, user, usr_prj)
            # Assume `get_user_projects` updates `usr_prj[user]` to be the list of projects
    # We have to block until all projects are listed before we can delete them.
    async with trio.open_nursery() as n:
        for prj in itertools.chain.from_iterable(usr_prj.values()):
            n.start_task(delete_project, prj)
    # And again, we have to be sure all projects are gone before we start on the users
    results = []
    async with trio.open_nursery() as n:
        for user in users:
            n.start_task(delete_user, user, results)
            # Assume `delete_user` appends its result to the given list
    return tuple(results)

The concern

I’m fairly sure this will “work”. However, opening three nurseries in sequence like this somehow feels messy at best and at worst like a good way to under-utilize Trio.

Before writing this, I perused the docs one more time and considered using a memory channel to implement a queue for this. I’ve only done something similar with asyncio once before, and I did it wrong :stuck_out_tongue: Not that I’m opposed to learning to do it right, but before investing the time I wanted to see if I’m even on the right track going that direction, or if I should stick with what I have, or if there’s a third way I’ve not considered.

2

I found the third way. Perhaps unsurprisingly, it was more abstraction. I wrote a new async function that does all the steps on just one object, then ran that function in tasks for each object I want to delete.

I’m still curious if a queue would be a better solution…

1 Like