taskman: an initial prototype for a trio.Nursery-like per-task-scope-manager (maybe via a user defined single-yield-generator function?) 😎

[goodboy]

As per some discussion in the trio gitter and starting something
toward #22 and supporting our debugger crash mode from within
a trio-compat task nursery.

---

Idea: a trio.Nursery API for optionally allowing a user to define a per-task manager

But why?

Well if you wanted any of the following (some of which are implemented
in the module script in this patch):

• generally gain access to lower level trio.Nursery "internals" for any
  per-task oriented purpose.
• some way to wrap your own "task handle" so you could do things like
  waiting on a task to finish and get its result.
• allow hooking into each task's exit/teardown (phase) for the purposes
  of debugging crashes, (unexpected) cancels or just generally tracing
  the trio per task control flow.
• enable more granular task supervision strategies like one_for_one
  from erlang/elixir and friends.

---

ToDo:

• offer a task_manager context-manager-as-generator API such
  that the current TaskOutcome struct is not tied to the
  internals of the ScopePerTaskNursery.start_soon() implementation.

  • ideally the outcome: Outcome = yield (cs, ..) is delivered to
    the user defined mngr via a Outcome.send(mngr) call such that only
    raw trio (dependency) internal types are provided to the user for
    wrapping / adjusting.
    • NOTE: didn't use Outcome.send() bc it would send the underlying
      value, not the outcome instance; not sure if that's best?

• way better naming since most of this was whipped up on the
  first try and getting good names is probably going to be tricky 😂

  • @task_manager is maybe better? @task_scope_manager?
  • ScopePerTaskNursery is terrible.. how about TaskScopedNursery?
    • now i'm thinking you don't need much at all, TaskManagerNursery?
  • TaskOutcome.unwrap() seems wrong, despite it being an example of
    a user defined handle.
    • @Fuyukai suggested .wait_for_result() which is a lot more
      pythonic in terms of plain-ol'-simple-semantics 🏄
    • maybe .unwind() is more descriptive? as in unwinding the outcome
      from the task would imply you both have to:
      • wait for the result or error from the underlying task
      • retrieve that outcome's value or raise it's error by unwrapping/boxing?
      • stack unwinding is normally synonymous with the end of
        a function call and the subsequent popping of the stack ..?
    • the nixed above ends up going down the rabbit hole of describing
      an imperative lang in terms of lower level fp abstractions which
      is not only confusing but a bit moot: we don't need to hear math
      about stuff we can't change in (python and) the semantics of
      a method..:joy:

• test the waters in trio core for interest in this idea

  • had a decent discussion on it with the anyio author and njs but
    don't think they understood what i was proposing (hence this proto
    😂)
  • the main spot to start digging into how this might be implemented
    inside trio's scheduler would be about here:
    https://github.com/python-trio/trio/blob/master/trio/_core/_run.py#L1601
    since this is where the nursery-global cancel scope's CancelStatus
    is activated on the currently spawning task.
    • obviously it's going to require quite a bit of reworking to try
      and offer the generator-style @cancel_scope_manager thing
      because we'll need to call __enter__/__exit__ from two
      different Runner methods -> .spawn_impl() and .task_exited()
  • probably just eventually accept that no-one will comment on this
    PR (nor care?) and it'll get relegated to our trionics secret
    sauces pantry yet again 😂

• figure out if it's possible to swap in this open_nursery()
  for all uses of trio.open_nursery() when our users enable debug mode
  😂

  • pretty sure we can do it at appropriate spots during (sub)actor boot?
    • root: https://github.com/goodboy/tractor/blob/master/tractor/_root.py#L135
    • child: https://github.com/goodboy/tractor/blob/master/tractor/_entry.py#L97

Hopefully much more to come!

[goodboy]

TODO: show an second example where the cancel_scope_manager generator wraps the trio internals listed as attrs into this compound type and then that is what's returned from the .start_soon() call?

[goodboy]

I don't really know what the best API would be for doing this with .start()?

1. you don't really need start since you can always implement it by getting the child task to Event.set(), or even further use this MaybeOutcome thing from above.
2. how do you get the cancel_scope_manager yielded output in that case, since .start() would normally return whatever the child passed to status.started()?

• i.e. you can't do started = await n.start(blah) and expect started to be the outcome, cs stuff

3. maybe the child task could get access to its own cancel scope "stuff" from somewhere and then choose to status.started((outcome, cs)) it back to the caller?

[goodboy]

Huh, i guess we could just offer some kinda support for hooking into the started value returned by the task? that way the cancel_scope_manager could decide what to send back to the .start() caller with the default being the value that the child .started() ?

[goodboy]

huh, likely you could implement that as a double yield generator then?

I dunno what would be best for ensuring that other then decorator params and then async_fn checking inside .start() against that i guess?

[goodboy]

As already mentioned above in the comment on the method, probably a better name is warranted for this since you're actually first waiting for the outcome then unwrapping it's underlying value/result.

[goodboy]

scheduling sleep_then_err() results in an assertion error that is caught and handled in the pdbp REPL thanks to the cancel_scope_manager implementation which does this on std Exceptions 😎

Note, you could also catch trio.Cancelled or wtv else by changing the except line 🏄🏼

[goodboy]

wrong, this should be the type sig of whatever is yielded from the self.scope_manager obviously

[goodboy]

yeah as mentioned in the TODO, don't love this concat-ing of the results from the scope_manager(), but i think this goes away if we instead implement our own generator as @trio.cancel_scope_manager protocol since then the absolute value tuple yielded can be returned directly?

[goodboy]

Ahh wait, right.. so we do need to activate this here or can it be done in the user's @task_scope_manager thing?

[goodboy]

Do we need this allocated here?
See comment(s) below.

[goodboy]

Heh, guess we don't need much of a special name now?

PerTaskHookableNursery 😂

[goodboy]

maybe task_manager is a better name?

[goodboy]

needs to again be dynamically defined at runtime.. soo use the ParamSpec stuff?

[goodboy]

lul, missing f-string

[goodboy]

maybe just @trio.task_manager ?

[goodboy]

no, we don't need the nursery to allocate any cancel scope per task interally!

need to remove this!