Skip to content

Upstream 16522 - Refactor formatted raw SQL in unified_jobs result_stdout_raw_handle#587

Merged
cigamit merged 1 commit into
mainfrom
upstream16522
Jul 18, 2026
Merged

Upstream 16522 - Refactor formatted raw SQL in unified_jobs result_stdout_raw_handle#587
cigamit merged 1 commit into
mainfrom
upstream16522

Conversation

@cigamit

@cigamit cigamit commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Upstream Summary

Refactors UnifiedJob.result_stdout_raw_handle() in awx/main/models/unified_jobs.py to stop building PostgreSQL COPY SQL with f-strings.

The event-based stdout path previously interpolated table names, column names, job_created, and job ID directly into a query string (# nosql), which internal security scans flag as formatted raw SQL.

This change replaces that with psycopg.sql composables:

  • sql.Identifier() for table and column names
  • sql.Literal() for job_created and the parent FK value

Values are embedded via sql.Literal() rather than %s placeholders with a separate params list, because Django’s cursor.copy() only forwards the SQL statement to psycopg and does not pass bind parameters.

Behavior is unchanged: the method still streams stdout from job event tables via COPY ... TO STDOUT, ordered by start_line, with the same partitioned vs _unpartitioned_ table selection and max-bytes checks. Django ORM is intentionally not used here to avoid loading large stdout blobs into memory.

The functional test sqlite_copy mock in awx/main/tests/functional/conftest.py was updated to load stdout from event tables directly instead of parsing SQL strings.

Related: AAP-77740 (discovered during AAP-76181)

Issue type

Bug, Docs Fix or other nominal change

Component

Other

Before

sql = f"copy (select stdout from {tbl} where {created_by_cond}{self.event_parent_key}={self.id} and stdout != '' order by start_line) to stdout"  # nosql
with cursor.copy(sql) as copy:

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **Bug Fixes**
  * Improved job stdout streaming reliability by using safer, parameterized database streaming for consistent output retrieval.
  * Kept the existing stdout size limit behavior and maintained the previous fallback flow when necessary.

* **Tests**
  * Updated functional test mocks to better emulate database COPY/stream behavior, including handling optional parameters.
  * Refined mock output selection to follow available event data rather than relying on query text parsing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

@cigamit
cigamit requested a review from TheWitness July 17, 2026 21:44
@cigamit cigamit self-assigned this Jul 17, 2026
Copilot AI review requested due to automatic review settings July 17, 2026 21:44
@cigamit cigamit added enhancement New feature or request python Pull requests that update python code labels Jul 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors UnifiedJob.result_stdout_raw_handle() to avoid building PostgreSQL COPY ... TO STDOUT SQL via f-strings, replacing interpolated raw SQL with psycopg.sql composables for safer identifier/literal handling while preserving the streaming stdout behavior.

Changes:

  • Replaced formatted COPY SQL construction in result_stdout_raw_handle() with psycopg.sql.SQL/Identifier/Literal composables.
  • Updated the SQLite cursor.copy() functional-test mock to no longer depend on parsing a SQL string.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
awx/main/models/unified_jobs.py Builds the COPY (SELECT ...) TO STDOUT statement using psycopg.sql composables instead of f-strings.
awx/main/tests/functional/conftest.py Adjusts the SQLite copy() mock used by stdout-related functional tests to align with the new query construction approach.

Comment thread awx/main/tests/functional/conftest.py
Comment thread awx/main/tests/functional/conftest.py
@cigamit
cigamit merged commit 8e7ac42 into main Jul 18, 2026
1 check passed
@cigamit
cigamit deleted the upstream16522 branch July 18, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request python Pull requests that update python code

Development

Successfully merging this pull request may close these issues.

3 participants