Skip to content

Conversation

@uptownhr
Copy link

Summary

Fixes #3808

This PR fixes the != comparator in DataFilter to implement true "NOT IN" semantics when used with multiple values.

Problem

The != comparator was exhibiting unintuitive behavior:

  • Before: Passed on first mismatch (value != ANY item in array)
  • After: Passes only if NO matches found (value NOT IN array)

Example of the Bug

With comparator: "!=" and value: ["id1", "id2", "id3"]:

Event: {"brandId": "id1"}

  • Old behavior: PASSES (because "id1" != "id2" is true)
  • New behavior: FAILS (because "id1" IS in the array)

Changes

Code Changes

  • pkg/sensors/dependencies/filter.go:
    • Added special handling for NotEqualTo comparator in JSONTypeNumber case
    • Added special handling for NotEqualTo comparator in JSONTypeString case
    • Both now check ALL values before deciding (true NOT IN semantics)
    • Other comparators maintain existing behavior

Test Coverage

  • pkg/sensors/dependencies/filter_data_test.go:
    • Added 4 new test cases proving the fix:
      1. String filter with value IN array (should FAIL)
      2. String filter with value NOT IN array (should PASS)
      3. Number filter with value IN array (should FAIL)
      4. Number filter with value NOT IN array (should PASS)

Test Results

$ go test ./pkg/sensors/dependencies/...
PASS
ok  	github.com/argoproj/argo-events/pkg/sensors/dependencies	0.010s

All existing tests pass with no regressions.

Breaking Change Note

This is technically a breaking change as it modifies the behavior of the != comparator with multiple values. However:

  1. The previous behavior was essentially broken/unusable
  2. No test coverage existed for this scenario (suggesting low/no usage)
  3. The new behavior matches intuitive expectations

Implementation Details

The fix checks if comparator is NotEqualTo and if so:

  1. Scans through ALL values in the array
  2. Tracks if ANY match is found
  3. Only passes if NO match was found (true NOT IN)

For all other comparators, the existing logic is preserved (pass on first match).

@uptownhr uptownhr requested a review from whynowy as a code owner November 12, 2025 17:44
… values

The != comparator in DataFilter with multiple values was exhibiting
unintuitive behavior that made it practically unusable. Instead of
implementing true "NOT IN" semantics (value must not match ALL items
in array), it was implementing "not equal to ANY" (passes on first
mismatch).

This meant that with comparator: "!=" and value: ["id1", "id2", "id3"],
an event with "id1" would PASS (because "id1" != "id2" is true),
when it should FAIL (because "id1" IS in the array).

Changes:
- Modified pkg/sensors/dependencies/filter.go to check ALL values
  before deciding for NotEqualTo comparator
- Applied fix to both JSONTypeNumber and JSONTypeString cases
- Added 4 comprehensive test cases proving the fix works correctly
- All existing tests continue to pass with no regressions

The fix implements true NOT IN semantics:
- PASS: if value does not match ANY item in the array
- FAIL: if value matches any item in the array

Fixes argoproj#3808

Signed-off-by: James Lee <[email protected]>
@uptownhr uptownhr force-pushed the fix/data-filter-not-equal-comparator branch from 580f76c to 9590fe5 Compare November 12, 2025 17:47
@whynowy
Copy link
Member

whynowy commented Nov 18, 2025

Thanks for raising the PR. Couple of questions:

  1. Shall we introduce a new comparator for not in, like !in, !()?
  2. Can this problem be solved by using other filters like expr or script?

@uptownhr
Copy link
Author

uptownhr commented Dec 1, 2025

Thanks for raising the PR. Couple of questions:

  1. Shall we introduce a new comparator for not in, like !in, !()?
  2. Can this problem be solved by using other filters like expr or script?
  1. good question - I'm thinking given = already does in, it makes more sense to have != be not in.
  2. i am currently using expr to get around this - but it would have been nice if != worked as I imagined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: DataFilter != comparator with multiple values doesn't work as NOT IN

2 participants