pkg/cdi: fix cache refresh handling for filesystem events

[thaJeztah]

• stacked on pkg/cdi: fix, and cleanup some locks, and use RWMutex #345
• pkg/cdi: clean up watcher lifecycle handling and simplify refresh logic #358
• fixes pkg/cdi: clean up watcher lifecycle handling and simplify refresh logic #358 (comment)

pkg/cdi: simplify restoration of removed directory watches

Handle removal of watched Spec directories separately from restoring
missing watches.

Previously update() accepted an optional removed directory and marked it
as unwatched only after attempting to restore other missing watches. This
required callers to distinguish between normal updates and directory
removal, and caused update() to serve two separate purposes.

Add markRemoved() to update the watch state when fsnotify reports that a
watched directory was removed, then call update() unconditionally to
restore any watches that are currently missing. This also allows a
directory that was removed and recreated concurrently to have its watch
restored immediately.

This keeps update() focused on reconciling missing watches and removes
the special-case removed-directory argument.

pkg/cdi: watch for Create events on all platforms

Include fsnotify.Create in the event mask on all platforms instead of only
on macOS.

Spec files can be written atomically by creating a temporary file and
renaming it into place. On Linux, fsnotify reports the destination of such
a rename as a Create event. Without watching Create events, the cache may
miss the new Spec and only refresh because of an unrelated event for the
temporary file.

Watching Create events directly ensures that newly created or atomically
replaced Spec files trigger a cache refresh on all platforms.

pkg/cdi: handle fsnotify operations as a bitmask

fsnotify.Event.Op is a bitmask and can contain multiple operations, but the
watcher compared it directly against individual operation values.

As a result, combined events could bypass operation-specific handling, such as
filtering writes and creates by file extension or detecting removal of a
watched directory.

Mask the operations we are interested in and test the individual bits instead.

pkg/cdi: ignore unrelated filesystem events

The watcher filtered write and create events by file extension, but rename
and remove events for unrelated files would still trigger a refresh of the
CDI cache.

Apply the same Spec-file filtering to all relevant filesystem events, while
still processing events for configured Spec directories themselves.

This avoids unnecessary rescans when non-Spec files in watched directories are
renamed or removed, without interfering with handling of removed Spec
directories.

[thaJeztah]

Let me just push the branch I had locally for my own sanity; I think I had some work remaining (last commit), but started to loose track of splitting out to small PRs (for easier review).

[thaJeztah]

Ah, right; I needed a go bump for some bits;

Error: pkg/cdi/cache.go:354:16: stdversion: slices.Sorted requires go1.23 or later (module is go1.21) (govet)
  	return slices.Sorted(maps.Keys(c.devices))
  	              ^
  Error: pkg/cdi/cache.go:365:16: stdversion: slices.Sorted requires go1.23 or later (module is go1.21) (govet)
  	return slices.Sorted(maps.Keys(c.specs))
  	              ^

[thaJeztah]

This is what previously caused the failures; my changes exposed this issue.

Had to (for testing) update to fsnotify 1.8, which added FSNOTIFY_DEBUG for debugging the events

[thaJeztah]

@klihub @elezar @bart0sh this is the last set of patches I had lined up 😅

Had to (for testing) update to fsnotify 1.8, which added FSNOTIFY_DEBUG for debugging the events

Slightly wondering if v1.7 is too conservative, and if we should bump to a slightly newer version, but we can leave that to consumers of this module.

[klihub]

LGTM

[elezar]

Slightly wondering if v1.7 is too conservative, and if we should bump to a slightly newer version, but we can leave that to consumers of this module.

I'd be ok with bumping to v1.8 as well. Did you want to do it in this PR, or as a separate dependency bump?

[klihub]

Slightly wondering if v1.7 is too conservative, and if we should bump to a slightly newer version, but we can leave that to consumers of this module.

I'd be ok with bumping to v1.8 as well. Did you want to do it in this PR, or as a separate dependency bump?

Let's do that separately later if we decide to do so. I think @thaJeztah only needed that for devel/debugging while working on this PR. No other pressing reasons for the bump.

[thaJeztah]

I'd be fine keeping it at 1.7, sticking to MVS and leaving it to consumers to update if they want. (although probably not many will be on such old versions).

[thaJeztah]

(I tried newer versions ( > 1.6) last week to see if they had a meaningful impact on indirect dependencies, but they didn't, so I picked 1.7 as a minimum)