Add blog post on GPU ecosystem

[gdalle]

This idea was discussed a while ago on Slack with @rayegun, so I got it started. Right now I'm just curious whether I forgot any relevant packages. Once we agree on the list, I'll add a small blurb on each item.

Ping @simeonschaub @kshyatt @maleadt

[SimonDanisch]

If you want to mention the new experimental vulkan backend, it's pretty far in terms of functionality:
https://github.com/SimonDanisch/Lava.jl
I will release it properly in the coming months, but it already compiles pretty complex code to pretty fast GPU binaries :)

[ejmeitz]

@krasow and I have been working on cuNumeric.jl which wraps and extends NVIDIA's cupynumeric library. The high-level goal is match the CUDA.jl array programming interface, but execute across distributed systems.

We already have feature parity with cupynumeric, and are well on our way to automatic kernel fusion which increases performance beyond what was possible in Python. We are hoping to announce the first stable release of the package this summer. Whether the lack of a release/maturity prevents cuNumeric.jl from being on this list I will leave to others to decide. Just wanted to self-promote :).

[gdalle]

I wrote the text with Claude and checked it manually for the packages I knew well. However, I'd love some proper feedback from the package authors on their respective paragraphs, so I'm allowing myself a bit of savage tagging here (sorry):

• CUDA and a bunch of other stuff: @maleadt, @kshyatt
• cuDSS: @amontoison, @michel2323
• cuNumeric: @ejmeitz
• AMDGPU: @jpsamaroo
• Metal: @christiangnrd
• OpenCL: @jakebolewski
• Vulkan: @serenity4
• Lava, GPUSelect: @SimonDanisch (what is the difference between Lava and Vulkan?)
• GPUEnv: @hakkelt
• Juliana: @101001000
• KernelAbstractions: @vchuravy
• KernelIntrinsics, KernelForge: @epilliat
• AcceleratedKernels: @anicusan
• JACC: @williamfgc, @PhilipFackler
• Tullio, TensorCast: @mcabbott
• OMEinsum: @GiggleLiu, @under-Peter
• TensorOperations: @Jutho, @lkdvos
• Reactant, ReactantCore: @wsmoses, @avik-pal, @mofeing

[serenity4]

I believe Vulkan.jl and OpenCL.jl should be moved to the Hardware-agnostic -> Low-level kernels section, and Lava.jl to the Hardware-agnostic -> High-level kernels section.

In particular, Vulkan.jl and OpenCL.jl are both meant to be hardware- and platform-agnostic, it's just that they have extensive concepts of optional feature/capabilities that specific hardware may decide to support or not. But for users of these two APIs should not really care about the specific GPU vendor (NVIDIA, AMD...).

On Lava.jl, AFAIU it is the high-level KernelAbstractions.jl interface implementation for Vulkan, sitting on top of Vulkan.jl with good old GPUCompiler magic.

[williamfgc]

Thanks for taking the initiative, nice job! I added some notes regarding JACC.jl as requested.

[gdalle]

Thanks @serenity4, i'll wait for @SimonDanisch to confirm and then move stuff around

[SimonDanisch]

Ah yes, @serenity4 points it out correctly ;)
Vulkan.jl to Lava.jl is almost like CUDACore.jl to CUDA.jl (although its really hard to compare those).
The cool thing about Lava.jl compared to AMDGPU.jl/CUDA.jl is, that on top of the normal KernelAbstraction interface, it also supports accessing the hardware raytracing acceleration primitives as well as compiling real graphics kernel from julia functions for the GPU, which is pretty unique!

[hakkelt]

Thanks for including my package in the post! I made some suggestions to make the description a bit more informative, but I was okay with the original as well.

[rayegun]

I think perhaps the environment switching like GPUEnv.jl / autodetection should go below KA.jl and the performance portability packages since they (or at least a few of them) reference and rely on KA.jl.

[mofeing]

Also mentioning @Pangoraw @jumerckx @glou-nes for comments.

[mofeing]

Suggested change

A companion sub-package, ReactantCore.jl, exposes the minimal type hierarchy needed by other packages to be Reactant-aware, allowing the broader Julia ecosystem to interoperate with Reactant's compilation pipeline.

[mofeing]

I would remove this because ReactantCore.jl is more like an implementation detail that is required for extension by other packages, just like what ChainRulesCore.jl is for ChainRules.jl. But I'm leaving it up to the writers.

[gdalle]

I think it's interesting to point out the possible integration without heavy deps

[mofeing]

Okay, but then I would fix the phrasing because ReactantCore.jl is not about the type-hierarchy but about the @trace macro.

Suggested change

	A companion sub-package, ReactantCore.jl, exposes the minimal type hierarchy needed by other packages to be Reactant-aware, allowing the broader Julia ecosystem to interoperate with Reactant's compilation pipeline.
	Reactant.jl tries to be as least intrusive as possible, but operator tracing may run into problems with control-flow constructs.
	A companion sub-package, ReactantCore.jl, exposes the `@trace` macro, which correctly marks control-flow constructs during tracing. The `@trace` translates to a noop if evaluated outside of the Reactant compilation context, allowing Reactant integration of the broader Julia ecosystem without fully depending on Reactant.

opinions @Pangoraw @jumerckx @glou-nes? since you guys have worked more with this