Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,32 @@ The backend package layout is feature-oriented:
- `pkg/ai` owns provider configuration, chat handling, tool definitions,
interaction pauses, Kubernetes tool execution, and tool authorization.

### AI request budgets

Two settings control model behaviour and they are not interchangeable:

- `AIMaxTokens` is a per-response ceiling. On current Claude models thinking and
answer text share it, so a small value truncates the answer. It is sent to the
provider as configured — never clamped, floored, or rejected, because only the
provider knows the configured model's real limit.
- `AIEffort` (`output_config.effort`) is the reasoning-depth knob and the only
one: `budget_tokens` is removed on current models and returns 400. Levels are
`low`/`medium`/`high`/`xhigh`/`max`, default `xhigh`. Anthropic path only.

`anthropicModelSupportsModernFeatures` gates effort, adaptive thinking, and
context management behind a deny list of model-name substrings. That list tracks
*request-surface support*, not lifecycle — Opus 4.5, Sonnet 4.5, and Haiku 4.5
are all still sold but reject the modern surface, and retired first-party models
stay listed because they remain available through Bedrock and Google Cloud. A
false negative here silently downgrades a capable model; there is no retry on a
400, so widening the gate needs a fallback path first.

SSE streams (`newStreamSender` in `pkg/ai/handler.go`) emit a keepalive comment
every 20s. An agent turn is legitimately silent while a tool runs, and
ingress-nginx closes a connection after 60s of backend silence. Chart timeouts
and the ingress annotation examples in `charts/kite/values.yaml` are the other
half of this — change them together.

## Request flow

Most protected API calls go through:
Expand Down
2 changes: 1 addition & 1 deletion charts/kite/templates/gateway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ spec:
- name: {{ include "kite.fullname" $ }}
port: {{ $.Values.service.port }}
timeouts:
request: {{ if .timeouts }}{{ .timeouts.request | default "120s" }}{{ else }}"120s"{{ end }}
request: {{ if .timeouts }}{{ .timeouts.request | default "900s" }}{{ else }}"900s"{{ end }}
{{- end }}
{{- end }}
{{- end }}
15 changes: 14 additions & 1 deletion charts/kite/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,16 @@ service:
ingress:
enabled: false
className: "nginx"
# ingress-nginx defaults will cut off AI chat and terminal streams: it closes a
# connection after 60s with no bytes from the backend, and rejects bodies over
# 1m. An SSE agent turn is legitimately silent while a tool runs, and the chat
# request carries the whole transcript. When enabling ingress, replace the
# empty map below with these annotations:
# annotations:
# nginx.ingress.kubernetes.io/proxy-read-timeout: "900"
# nginx.ingress.kubernetes.io/proxy-send-timeout: "900"
# nginx.ingress.kubernetes.io/proxy-body-size: "32m"
# nginx.ingress.kubernetes.io/proxy-buffering: "off"
annotations: {}
hosts:
- host: kite.zzde.me
Expand Down Expand Up @@ -317,7 +327,10 @@ gateway:
type: PathPrefix
value: /
timeouts:
request: "120s"
# An AI agent turn streams over SSE for as long as the model keeps
# calling tools, which routinely exceeds two minutes. A shorter
# timeout cuts the stream mid-answer with no error the user can act on.
request: "900s"
# Example: additional route for different hostname/path
# - name: kite-api
# annotations: {}
Expand Down
Loading