Skip to content

Fix XCom get_value outside task runner#68789

Open
sdhilip200 wants to merge 1 commit into
apache:mainfrom
sdhilip200:codex/fix-xcom-extra-link-context
Open

Fix XCom get_value outside task runner#68789
sdhilip200 wants to merge 1 commit into
apache:mainfrom
sdhilip200:codex/fix-xcom-extra-link-context

Conversation

@sdhilip200

@sdhilip200 sdhilip200 commented Jun 21, 2026

Copy link
Copy Markdown

Plugin-registered extra links are resolved in the API server, but examples can naturally reach for XCom.get_value() from the SDK/compat SDK. In Airflow 3 that SDK XCom path is task-runner scoped, so it expects SUPERVISOR_COMMS and is not suitable for API-server extra-link code.

This adds a server-side XComModel.get_one() / XComModel.get_value() convenience path that reuses the existing metadata DB query and deserialization behavior from XComModel. The common compat SDK now exposes XComModel, so plugin extra links can use an API-server-safe XCom reader without importing task-runtime SDK XCom.

This also adds route coverage showing a plugin-registered extra link resolving through /links and reading an XCom value with XComModel.get_value().

Closes #59093

Tests:

  • uv run pytest airflow-core/tests/unit/models/test_xcom.py
  • uv run pytest airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_extra_links.py
  • uv run pytest --rootdir=providers/common/compat providers/common/compat/tests/unit/common/compat/test_sdk.py
  • uv run pytest task-sdk/tests/task_sdk/bases/test_xcom.py
  • uv run ruff check airflow-core/src/airflow/models/xcom.py airflow-core/tests/unit/models/test_xcom.py airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_extra_links.py providers/common/compat/src/airflow/providers/common/compat/sdk.py providers/common/compat/tests/unit/common/compat/test_sdk.py task-sdk/src/airflow/sdk/bases/xcom.py
  • uv run ruff format --check airflow-core/src/airflow/models/xcom.py airflow-core/tests/unit/models/test_xcom.py airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_extra_links.py providers/common/compat/src/airflow/providers/common/compat/sdk.py providers/common/compat/tests/unit/common/compat/test_sdk.py task-sdk/src/airflow/sdk/bases/xcom.py

Was generative AI tooling used to co-author this PR?
  • Yes: OpenAI Codex

Generated-by: OpenAI Codex following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

@boring-cyborg

boring-cyborg Bot commented Jun 21, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@sdhilip200 sdhilip200 marked this pull request as ready for review June 21, 2026 03:17
@justinpakzad

Copy link
Copy Markdown
Contributor

Hi, I don't think this is the right approach. The task-sdk should not be accessing the database directly. I would take a look at the docs here for a bit more detail on database access restrictions:
https://airflow.apache.org/docs/apache-airflow/stable/installation/upgrading_to_airflow3.html#database-access-restrictions

@sdhilip200 sdhilip200 force-pushed the codex/fix-xcom-extra-link-context branch from cc8f502 to cee3476 Compare June 21, 2026 22:00
@sdhilip200 sdhilip200 requested a review from XD-DENG as a code owner June 21, 2026 22:00
@sdhilip200

Copy link
Copy Markdown
Author

Updated the PR in cee3476 to avoid DB access from task-sdk. The revised patch restores BaseXCom to the supervisor-comms-only path, adds server-side XComModel.get_one()/get_value() helpers, exposes XComModel from the common compat SDK for plugin extra links, and adds /links route coverage for a plugin-registered extra link reading XCom through XComModel.

@Vamsi-klu

Copy link
Copy Markdown
Contributor

This gives sdk.XComModel.get_value() different semantics between Airflow 2 and 3. On Airflow 2 the compat fallback resolves through XCom.get_value() and the configured XCom backend, but this Airflow 3 helper always calls XComModel.deserialize_value(). With object-storage/custom XCom backends, extra-link code can receive the stored reference/path instead of the actual value. Please either route deserialization through the configured backend for this server-side helper, or make the raw-DB semantics explicit and add a regression test with XComObjectStorageBackend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

XCom.get_value() via SDK fails in extra link plugin with SUPERVISOR_COMMS ImportError

3 participants