Skip to content

blog: SQLite vs DuckDB benchmark + ingest admission gate#268

Open
jstojiljkovic wants to merge 17 commits into
mainfrom
feature/duckdb-benchmark-blog
Open

blog: SQLite vs DuckDB benchmark + ingest admission gate#268
jstojiljkovic wants to merge 17 commits into
mainfrom
feature/duckdb-benchmark-blog

Conversation

@jstojiljkovic

@jstojiljkovic jstojiljkovic commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Benchmark series post 3: SQLite vs DuckDB telemetry backends on the same Hetzner CCX13.

  • website: new post sqlite-vs-duckdb.mdx plus read-probe charts and dashboard screenshots. Every number in the post is backed by the raw JSONs in this PR.
  • backend: ingest admission gate (INGEST_MAX_CONCURRENT, INGEST_ADMISSION_WAIT_SECONDS) that caps concurrent ingest processing and sheds overload with 503 + Retry-After instead of getting OOM-killed.
    Rejects are counted and exposed as ingestRejected in /api/health/deep.
  • loadgen: post-fill digestion gate (--max-digest-wait) so read probes measure steady-state query cost rather than a mid-WAL-checkpoint engine; fills now count only 2xx-confirmed rows so admission 503s
    don't produce phantom rows; workflow dispatch overrides for the read threshold and the Phase 2 rate ladder.
  • data: throughput, read-probe, and 60-second diagnostic JSONs per signal in benchmarks/blog/post-3-data/.

@jstojiljkovic jstojiljkovic self-assigned this Jul 21, 2026
@jstojiljkovic
jstojiljkovic marked this pull request as ready for review July 22, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant