Feat: Improved IO handling for YARA and hot-reload

[esoadamo]

Summary

This PR addresses unnecessary IO load when running on slower HDDs. It performs thee changes:

• it disables YARA matching if no YARA rules are loaded, lowering the IO load
• it allows having empty YARA/Sigma rules folder (currently when a folder was empty, the reload was denied), which allows for the YARA rules to be empty once at least one rule was present
• it changes from polling the FS every 2 seconds for rules change into FS watch that fires on change (and fallbacks to 60 seconds poll if setting up watches fails), which allows the HDD to go to sleep if there are no changes

Type of change

• feat / enhancement - new feature
• bug - bug fix
• refactor - refactoring, no behaviour change
• documentation - docs only
• ci - CI/CD changes
• dependencies - dependency update

Test plan

• Tested on Windows
• Tested on Linux
• Existing tests pass (cargo test)
• New tests added (if applicable)

Checklist

• Label added to this PR
• Docs updated (if behaviour changed)

[Karib0u]

Thanks for the PR. The problem is real: the old hot-reload poller fingerprints rule/IOC files every 2s by default, so replacing that with filesystem events makes sense.

I would ask for a few changes before merging:

1. Empty folders vs broken rule files need to be distinguished.

Allowing an intentionally empty Sigma/YARA folder is fine, but right now a folder containing only invalid rules can also rebuild to zero loaded rules and replace the last good detector. That could silently disable detection after a bad rule deploy. I think reload should allow “no rule files found”, but reject “rule files found and none compiled/loaded”.

2. The fallback poll interval does not match the PR/docs.

The PR says watcher failure falls back to a 60s poll, but the code polls as fast as max(debounce_ms, 50ms) when debounce_ms < 2000. If someone sets a small debounce and watcher setup fails, this can create much more IO than before. Fallback should probably be fixed at 60s, with any fast interval kept test-only or separately configurable.

3. IOC file watching should probably watch parent directories.

The PR watches IOC files directly. Some editors/config tools update files via temp-file-plus-rename, which can make direct file watches unreliable. Watching the parent directory non-recursively and keeping the existing fingerprint comparison would be more robust.

4. config.toml still has the old reload comment.

It still says poll cadence is max(debounce_ms, 2000), which no longer describes the new watcher behavior.

Overall: good direction and the core issue is valid, but I would not merge as-is until at least points 1 and 2 are fixed.

[esoadamo]

Thanks for the review! I have addressed all points:

1. Empty folders vs broken rule files are distinguished during reload:

   • Modified Engine and Scanner to track the count of rule files found in the directory (rule_files_found / files_found) and how many compiled successfully (rule_count / compiled_files).
   • Replaced returning an error from Scanner::new and Engine::load_rules with warnings and soft-tracking of failed files. This ensures bad rules do not fail daemon initialization or crash the process.
   • In the hot reload worker (spawn_reload_worker in src/reload/mod.rs), if all files found in the directory are broken (none compiled, e.g., files_found > 0 but compiled_files == 0), the reload is rejected entirely, printing a warning and keeping the previous valid configuration active.
   • If some rules are working and some are broken, the reload is accepted: working rules are successfully swapped in, while warning details for the broken rules are printed to the logs.
   • Fully empty directories (no files found) are still allowed to reload into an empty state (clearing active rules).

2. The fallback poll interval matches the PR/docs:

   • Standardized the fallback polling cadence to a fixed 60 seconds for production environments in spawn_reload_poller in src/reload/mod.rs.
   • Fast polling fallback is restricted to test environments via cfg!(test) to ensure integration tests continue to run quickly.

3. IOC file watching watches parent directories:

   • Updated the notify watcher setup in spawn_reload_poller to watch the unique parent directories of the configured IOC files non-recursively, rather than watching the files directly.
   • Together with the fingerprint checks, only changes to the specific IOC files themselves will trigger a reload, preventing editor temp-file-plus-rename events from breaking the file watch.

4. config.toml reload comment:

   • Updated the comment for debounce_ms in config.toml to accurately describe the filesystem events and 60s fallback polling.