Skip to content

Add CI that checks Snowflake and Trino share the Iceberg table - #3

Merged
HarshCasper merged 2 commits into
mainfrom
ci/assert-iceberg-interop
Oct 14, 2026
Merged

HarshCasper merged 2 commits into
mainfrom
ci/assert-iceberg-interop

Conversation

@HarshCasper

@HarshCasper HarshCasper commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

The repo had no CI, and nothing checked that the demo works.

  • New test.sh (make test) walks through the demo on a freshly started stack:
    • the create-polaris-catalog container exits 0, and the Polaris API returns the polaris catalog with base location s3://test-bucket/test
    • creates test-bucket, then runs script.sql statement by statement on LocalStack for Snowflake through snowflake-connector-python, and checks the final SELECT returns foobar and test
    • Trino returns the same rows from iceberg.test_namespace.iceberg_table
    • s3://test-bucket/test/ holds Parquet data files and Iceberg metadata.json files
  • create-polaris-catalog.sh now checks HTTP status codes. Before, curl -i without --fail printed "Catalog created successfully" even when Polaris rejected the request. A 409 (catalog exists) is accepted, so the script can run again.
  • make run ran the setup script on the host, where http://polaris:8181 does not resolve. It now runs the compose service. make check looks for the AWS CLI instead of lstk, which the demo does not use.
  • docker-compose.yml required LOCALSTACK_AUTH_TOKEN for every compose command, so make stop, make logs and docker compose exec failed unless the token was exported. It now defaults to empty, and make start still checks that it is set.
  • README: test instructions, the catalog setup that make start already runs, docker compose instead of docker-compose, and the LocalStack health URL.
  • New workflow: starts the stack with make start, runs make test, prints container logs on failure.

Verified by CI.

https://claude.ai/code/session_01DZfDmzUBPqNAu3yFKyfCNM

test.sh runs script.sql on LocalStack for Snowflake, reads the table back from Trino through Polaris and checks the Iceberg files in S3. The catalog setup script now fails on HTTP errors, and make run uses the compose service, since polaris:8181 only resolves inside the compose network.

Claude-Session: https://claude.ai/code/session_01DZfDmzUBPqNAu3yFKyfCNM
docker-compose.yml required LOCALSTACK_AUTH_TOKEN, so make stop, make logs and docker compose exec failed unless the token was exported. make start still checks it.

Claude-Session: https://claude.ai/code/session_01DZfDmzUBPqNAu3yFKyfCNM
@HarshCasper
HarshCasper merged commit 828764b into main Oct 14, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant