Skip to content

NIFI-7686: Support multiple key columns in database lookup services#11188

Open
lhapollo wants to merge 1 commit intoapache:mainfrom
lhapollo:NIFI-7686
Open

NIFI-7686: Support multiple key columns in database lookup services#11188
lhapollo wants to merge 1 commit intoapache:mainfrom
lhapollo:NIFI-7686

Conversation

@lhapollo
Copy link
Copy Markdown

@lhapollo lhapollo commented Apr 29, 2026

Summary

NIFI-7686

Added composite key support to DatabaseRecordLookupService and SimpleDatabaseLookupService. When LOOKUP_KEY_COLUMN contains a comma-separated list of column names, the services build a composite WHERE clause and expect coordinates keyed by column name instead of the single 'key' coordinate.

Includes type coercion so String values from LookupRecord are converted to the correct SQL types based on ParameterMetaData.

Backward compatible: single-key behavior is unchanged.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

@dan-s1
Copy link
Copy Markdown
Contributor

dan-s1 commented Apr 29, 2026

@lhapollo Please note the merge is blocked as the commits must have verified signatures. Please see here for creating commits with verified signatures

Added composite key support to DatabaseRecordLookupService and
SimpleDatabaseLookupService. When LOOKUP_KEY_COLUMN contains a
comma-separated list of column names, the services build a composite
WHERE clause and expect coordinates keyed by column name instead of
the single 'key' coordinate.

Includes type coercion so String values from LookupRecord are
converted to the correct SQL types based on ParameterMetaData.

Backward compatible: single-key behavior is unchanged.
@dan-s1
Copy link
Copy Markdown
Contributor

dan-s1 commented May 1, 2026

@lhapollo It looks like there are Checkstyle and PMD violations in the two builds that are failing. Please correct those.

Copy link
Copy Markdown
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for proposing this new feature @lhapollo. On initial review, this change introduces notable behavior changes, and has duplicative logic for things like coercing values. Expanding the semantic behavior of the Lookup Key Column property is also potentially confusing, so at minimum, it would require renaming the property. For all of these reasons, I think the approach needs closer consideration. It may warrant a different service implementation, rather than changes to the existing implementations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants