-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Fixes #24551: Add catalogId support for Athena S3 Tables and cross-account Glue catalogs #25356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
875dd50 to
9e074a2
Compare
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Code Review ✅ ApprovedClean feature addition that consistently propagates the optional catalogId parameter across all relevant Athena/Glue API calls with proper defensive checks. What Works WellThe implementation properly handles the optional catalogId throughout the codebase, using consistent conditional checks before adding the parameter to API calls. URL encoding is correctly applied in the connection URL, and the documentation clearly explains the use cases for cross-account and S3 Tables scenarios. OptionsAuto-apply is off Gitar will not commit updates to this branch. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | This comment will update automatically (Docs) |
|
Cross account s3 tables catalog not tested, only same account tested and working. |
…ue catalogs Add catalogId parameter to the Athena connector to enable querying databases from S3 Tables catalogs and cross-account Glue Data Catalogs. Changes: - Add catalogId property to athenaConnection.json schema - Pass catalog_name to pyathena connection URL for database listing - Pass CatalogId to Glue API calls (get_databases, get_table) - Pass CatalogId to Lake Formation tag APIs - Add field documentation Closes open-metadata#24551
9e074a2 to
10a7b94
Compare
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Describe your changes:
Fixes #24551
Add
catalogIdparameter to the Athena connector to enable querying databases from S3 Tables catalogs and cross-account Glue Data Catalogs.Changes:
catalogIdproperty toathenaConnection.jsonschemacatalog_nameto pyathena connection URL for database listingCatalogIdto Glue API calls (get_databases,get_table)CatalogIdto Lake Formation tag APIsTesting:
Tested locally with an S3 Tables catalog, ingestion ran fine.
Note on migrations: No migration needed -
catalogIdis a new optional field with no default value. Existing connections will continue to work without it.Type of change:
Checklist:
Fixes <issue-number>: <short explanation>