Cloud GPU

Burst capability for coding agents: when a task is too big for your local GPU, spin up a cloud GPU on demand, serve a large model (Qwen/Qwen3-32B-FP8) through vLLM behind an OpenAI-compatible API, point a coding agent at it, then tear it down to stop billing.

  make up  ──►  Runpod pod (1x H100 / RTX PRO 6000)
                  └─ vLLM ─ https://<pod-id>-8000.proxy.runpod.net/v1
                              ▲
                    OpenCode ─┘  (or aider, or any OpenAI-compatible client)
  make down ─►  pod terminated, billing stops

This costs real money. A pod runs $2–3.50/hr, billed per second from boot, and a network volume bills continuously whether or not a pod is attached. make down is the only thing that reliably stops the meter — read Billing and how you get burned before your first run.

Everything below is a from-scratch setup. Expect ~20 minutes of one-time work, then ~10 minutes per launch.

---

Contents

1. Prerequisites
2. Set up Runpod
3. Set up this repo
4. Your first make up
5. Set up OpenCode on your workstation
6. Everyday workflow
7. Command reference
8. Billing and how you get burned
9. Troubleshooting

---

1. Prerequisites

On the machine that will run the scripts (Linux or macOS):

Tool	Why	Check
bash 4+	the scripts	bash --version
curl	Runpod API calls	curl --version
jq	JSON parsing	jq --version
git, make	clone and drive	make --version
node 18+ / npm	OpenCode (step 5)	node --version

Install the missing ones:

sudo apt install -y curl jq git make      # Debian/Ubuntu
brew install curl jq git make             # macOS

You also need a Runpod account with a payment method — that's step 2.

---

2. Set up Runpod

2.1 Create the account

1. Go to runpod.io and sign up (Google/GitHub SSO or email).
2. Verify your email and log in to the console.

2.2 Add credit

Runpod is prepaid — pods will not start on a $0 balance.

1. Open Billing in the console sidebar.
2. Add a payment method, then Add Credit. The minimum is $10, which is a sensible starting point: it covers roughly 3–4 hours of H100 time.
3. While you're there, set a spending alert if the option is available to you.

Do not enable auto-reload until you trust your own teardown habits.

2.3 Create a network volume

The volume is persistent disk that caches the ~33GB of model weights between pods. Without it, every launch re-downloads them (5–10 extra minutes, paid at GPU rates).

1. Open Storage → New Network Volume.
2. Datacenter: this is the one irreversible choice here — a pod can only mount a volume in the volume's own region, so the region you pick constrains which GPUs you can ever get. US-NE-1 is the measured-good default (see docs/timings.md); if you pick another, check that it stocks H100 or RTX PRO 6000 Blackwell.
3. Size: 50 GB. That holds one 32B FP8 model plus headroom.
4. Name: anything (vllm-cache works).
5. Create it.

Cost is roughly $0.07/GB/month — about $3.50/month for 50GB, billed continuously, pod or no pod. Delete the volume if you stop using this project.

You do not need to note the volume ID: if the account has exactly one network volume, the scripts find and attach it automatically. (With more than one, you'll set NETWORK_VOLUME_ID in config.sh.)

2.4 Generate an API key

1. Open Settings → API Keys → Create API Key.
2. Give it a name (cloud-gpu) and read/write permission — it needs to create and delete pods.
3. Copy the key now. Runpod shows it exactly once.

---

3. Set up this repo

git clone git@gitlab-ssh.home.d-tork.dev:helpers/cloud-agent.git cloud-gpu
cd cloud-gpu

3.1 Add your API key

echo 'RUNPOD_API_KEY=rpa_your_key_here' > .env
chmod 600 .env

.env is gitignored. Never paste this key into a commit, an issue, or a log.

3.2 Verify it works

make check     # syntax-checks every script, touches nothing
make status    # hits the API — should say "no live pods — nothing is billing"

If make status reports an auth failure, the key is wrong or lacks write permission.

3.3 Skim the config

config.sh holds every tunable, and every value can be overridden from the environment for a single run. The ones worth knowing before your first launch:

Setting	Default	What it does
MODEL	Qwen/Qwen3-32B-FP8	Needs ≥80GB VRAM and Hopper-or-newer. On Ampere, switch to bf16 Qwen/Qwen3-32B.
GPU_CANDIDATES	H100 SXM, RTX PRO 6000, …	Ordered preference list. Deploys the first one actually in stock.
MAX_HOURLY_USD	3.50	Hard price ceiling. Pricier candidates are skipped, never silently used.
MAX_HOURS	4	Wall-clock auto-kill.
IDLE_MINUTES	60	Auto-kill after this long with no inference activity.
RETRY_MINUTES	15	How long to keep hunting for capacity. Nothing bills while it waits.

Lower MAX_HOURS and IDLE_MINUTES if you're nervous. They cost you nothing but a restart.

---

4. Your first make up

make up

It prints a summary and waits for you to confirm before anything bills:

  Provisioning a burst GPU
  ---------------------------------------------------------------
  Model      Qwen/Qwen3-32B-FP8
  Image      vllm/vllm-openai:v0.27.0
  Storage    vllm-cache (olx3mldxvt, 50GB) at /runpod-volume
  Region     US-NE-1 (pinned by the volume)
  Candidates (deploys the first one in stock, SECURE):
    LOW     $3.29   1x H100 80GB HBM3 80GB
    LOW     $2.09   1x RTX PRO 6000 Blackwell 96GB
    NONE    $2.79   1x H100 NVL 94GB
  Cost       up to $3.29/hr, billed per second from boot
  Auto-kill  after 4h (max $13.16) or 60min idle,
             enforced by a watchdog on THIS machine only
  Retry      keeps re-checking stock for up to 15min
  ---------------------------------------------------------------

  Start it? [y/N]

Answer y. From here it hunts for capacity, deploys, and polls until the model is loaded.

Your first launch is a cold start: budget ~15 minutes (the weight cache is empty). Later launches are warm — still ~8–9 minutes, because the 25GB container image is pulled fresh every time. Don't cycle the pod casually; see docs/timings.md.

When it's ready you get:

  Endpoint   https://kc3lhnjn8eviid-8000.proxy.runpod.net/v1
  Model      Qwen/Qwen3-32B-FP8
  API key    sk-1a2b3c...

  Smoke test:   make chat
  Watch spend:  make status
  Stop billing: make down

There are now two different keys in play. Keep them straight:

• RUNPOD_API_KEY in .env — your Runpod account key, controls billing.
• VLLM_API_KEY in .pod — a random per-pod bearer token, printed above, regenerated on every make up. This is the one your coding agent uses. It exists because the Runpod proxy URL is public to anyone who learns the pod id.

Confirm the endpoint answers:

make chat

Then, for the rest of this section, tear it back down so you're not paying while you install OpenCode:

make down

make down terminates the pod (not "stop" — terminate) and reports what the run cost.

---

5. Set up OpenCode on your workstation

OpenCode is a terminal coding agent. This is a one-time install plus a config file you'll edit the pod id in on each launch.

5.1 Install

npm i -g opencode-ai
# or:  curl -fsSL https://opencode.ai/install | bash

opencode --version

5.2 Register the Runpod provider

Create ~/.config/opencode/opencode.jsonc:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "runpod": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Runpod vLLM",
      "options": {
        "baseURL": "https://REPLACE-POD-ID-8000.proxy.runpod.net/v1",
        "apiKey": "{env:VLLM_API_KEY}"
      },
      "models": {
        "Qwen/Qwen3-32B-FP8": {
          "name": "Runpod cloud Qwen3 32B",
          "limit": { "context": 32768, "output": 8192 }
        }
      }
    }
  }
}

A copy with annotations lives at docs/opencode.jsonc.

Three things that produce a bare, unhelpful Not found:

1. baseURL must end in /v1 — vLLM serves nothing at the root.
2. The key under models must be the exact model id the server reports (Qwen/Qwen3-32B-FP8), not a friendly name.
3. context must not exceed the server's MAX_MODEL_LEN (32768 by default).

5.3 Connect it to a running pod

Every make up produces a new pod id and a new token, so both of these are per-launch steps:

make status     # prints the current endpoint URL

Edit the baseURL in ~/.config/opencode/opencode.jsonc to match, then export the token:

cd /path/to/cloud-gpu
export VLLM_API_KEY=$(. .pod && echo "$VLLM_API_KEY")

Now run the agent from whatever project you're working on:

cd ~/code/my-project
opencode

Inside OpenCode, use /models and pick Runpod vLLM → Runpod cloud Qwen3 32B.

Prefer aider? It needs no config at all — eval "$(make env)" exports exactly the variables it reads, then aider --model "openai/$OPENAI_MODEL". Details in RUNPOD.md.

---

6. Everyday workflow

Once everything above is done, a burst session is:

cd /path/to/cloud-gpu
make up                                              # ~8-9 min, prints the endpoint
export VLLM_API_KEY=$(. .pod && echo "$VLLM_API_KEY")
# update baseURL in ~/.config/opencode/opencode.jsonc with the new pod id

cd ~/code/my-project && opencode                     # work

cd /path/to/cloud-gpu && make down                   # STOP BILLING

Leave a make status in another terminal if you want to watch the spend climb.

---

Command reference

Command	What it does
make up	Provision the pod and wait for the model to load. Confirms cost first.
make up-yes	Same, without the confirmation prompt. Use deliberately.
make down	Terminate the pod and stop billing. Reports the run's cost.
make status	Every live pod on the account, $/hr, uptime, spend so far, watchdog liveness.
make gpus	Which candidate GPUs are in stock, where, and at what price. make gpus-all for the full catalog.
make chat	One-shot chat completion against the endpoint — a smoke test.
make env	Shell exports (OPENAI_BASE_URL, OPENAI_API_KEY, …) for OpenAI-compatible clients.
make logs	Recent pod logs. make follow tails them.
make check	Syntax-check every script. Touches no API.

scripts/down.sh <pod-id> terminates a pod this machine isn't tracking.

---

Billing and how you get burned

Four guardrails, and the hole in them:

1. Confirmation prompt. make up never bills without a y, and refuses to run non-interactively unless you pass -y.
2. Price ceiling. Candidates above MAX_HOURLY_USD are skipped, so falling down the preference list can't quietly cost more than the number you agreed to.
3. Duplicate guard. make up refuses to start if any pod is already live on the account, tracked or not.
4. Watchdog. A background process on your machine polls the pod every 60s and terminates it after MAX_HOURS or IDLE_MINUTES of no inference.

The hole: the watchdog dies with the machine it runs on. If your laptop sleeps, reboots, or loses power, the pod keeps billing until you come back. Runpod has no server-side idle timeout for pods — that's a Serverless-only feature. make status says loudly when the watchdog isn't running.

So: make down is the real control. Get in the habit. And when in doubt, the Runpod console is the source of truth for what is actually running.

Also remember the network volume bills continuously, independent of any pod.

---

Troubleshooting

make up says "no capacity" after 15 minutes. Popular GPUs sell out constantly. Run make gpus to see what's actually in stock in your region. Options: raise RETRY_MINUTES, add GPUs to GPU_CANDIDATES, raise MAX_HOURLY_USD, or run NETWORK_VOLUME_ID=none make up to search every region at the cost of re-downloading the weights.

make up refuses to start — "already RUNNING". Something is live and billing. make status to see it, make down (or scripts/down.sh <pod-id>) to kill it.

OpenCode returns a bare Not found. Almost always the baseURL (must end in /v1, must have the current pod id) or the model key. Re-check make status for the live endpoint.

Responses come back empty (content: null). Qwen3 is a reasoning model, and if max_tokens runs out before it closes its <think> block you get nothing back. It's also ~95x the tokens for a trivial answer. Disable thinking server-side for one run:

VLLM_EXTRA_ARGS='--reasoning-parser qwen3 --default-chat-template-kwargs {"enable_thinking":false}' make up

Edit VLLM_EXTRA_ARGS in config.sh to make it permanent. Runpod splits args on whitespace, so JSON values here must contain no spaces. Full measurements in docs/timings.md.

"Could not reach Runpod — cannot confirm what is running." The API is flaky sometimes. Read-only commands retry and then say this rather than claiming nothing is billing. Check the console.

---

Going further

• RUNPOD.md — how the provisioning works, the capacity hunt, the proxy security model, and the two known gaps (watchdog dies with the local machine; no SSH/Tailscale without a custom image).
• docs/timings.md — measured startup timings, costs, observed GPU availability, and the Qwen3 reasoning gotchas.
• config.sh — every tunable, commented.