Async Kotlin client in Java SDK

[uranusjr]

Java doesn’t do coroutines, so the current Client implementation is sync. But Kotlin does, and we might benefit from a different async client. I think we can do it like this:

1. Add an AsyncTask protocol that has an suspend fun execute instead of the sync execute function.
2. Create an AsyncClient class to use instead of Client. suspend fun execute(client: AsyncClient, context: Context).
3. When the task is executed, check which protocol is used. Wrap runBlocking around the async task.
4. Make the annotation processor detect suspend fun so the same @Builder.Task can be used on both sync and async and automatically does the right thing.

The only question is whether we just add Async to both Task and Client. Should the sync version be renamed? (BlockingTask & BlockingClient?)

[subramanya-dev]

@uranusjr I’d be happy to work on this feature, please assign it to me if available.

[potiuk]

We do not assign issues - see contributing guide. But feel free to work on it.

[uranusjr]

I thought about this a bit. Since it doesn’t really make much sense to expose this to Java (any language except Kotlin really), it’s probably better to split this into a separate package, or even a separate artifact. Not sure. Probably better than to prefix everything (AsyncClient AsyncTask and so on. I’m not exactly sure what the best approach here is tbh.)

Either way, we should

1. Create an async equivalent of org.apache.airflow.sdk.execution.Client.
2. Create an async equivalent of org.apache.airflow.sdk.Client.
3. Create an async Task interface that requires suspend fun execute(Context, Client) where Client is the async version (Context should be the same).
4. Extend Dag.add_task() to also accept an async Task, and TaskRunner to handle it.
5. Extend @Builder.Task to be able to convert an annotated suspend fun to an async Task.