Open source, bare metal to tokens: one platform to schedule, serve, benchmark and observe LLMs on NVIDIA and AMD GPUs.
Website: https://openinferencefabric.github.io
oif installs, with one command, a Kubernetes platform on GPU servers: open-source NVIDIA Run:ai and the KAI Scheduler for projects, quotas and fractional GPUs, NVIDIA Dynamo for serving, AIPerf for benchmarks, and Grafana dashboards for every GPU. NVIDIA and AMD GPUs run side by side in one cluster, and every step has a tested guide. Apache-2.0.
Our changes go back to the upstream projects, not into private copies. Everything below is open for review.
| Project | What we sent |
|---|---|
| Run:ai | PRs #3950 AMD GPUs in the node inventory · #3948 AMD icon for ROCm images · #3949 GPU fractions readiness; issue #4274 AMD GPU metrics |
| KAI Scheduler | PRs #2557 reservation pod resources · #2594 topology required level; design for AMD GPU fractions #2556 |
| KAI gpu-fractioning | PR #175 hard VRAM limit per pod on AMD GPUs (Linux dmem cgroup) |
| NVIDIA Dynamo | ROCm images for vLLM: co-developing #12537 (AMD) on branch rocm, image builds on vLLM 0.33.0, benchmark on the Radeon PRO W7900 next |
| ROCm | Issues rocm-systems#14860 VRAM limits in hipMemGetInfo · #14861 per-process GPU busy time · device-metrics-exporter#722 per-pod metrics for shared GPUs |
| KWOK | Issue #1802 AMD Instinct example nodes |
Every item, with lab evidence: docs/UPSTREAM.md · live status: https://openinferencefabric.github.io/upstream/.
- Test on NVIDIA, AMD or mixed GPU servers and tell us what breaks.
- Review the upstream pull requests above, or try the Dynamo ROCm branch on your AMD GPU.
- Contribute Go, Kubernetes, ROCm/CUDA or docs: start with CONTRIBUTING.md.