Make Sidecar default collector paths and host variable configurable

[xd4rker]

Description

Make the hardcoded paths and host-variable name in the default Sidecar collectors (seeded by the V20180212165000_AddDefaultCollectors migration) configurable via graylog.conf, for deployments with a custom Sidecar/server layout.

New server options, all defaulting to the current values:

• sidecar_collector_binary_dir (default: /usr/lib/graylog-sidecar)
• sidecar_spool_dir (default: /var/lib/graylog-sidecar)
• sidecar_windows_install_dir (default: C:\Program Files\Graylog\sidecar)
• sidecar_server_config_dir (default: /etc/graylog/server)
• sidecar_host_variable (default: graylog_host)

How Has This Been Tested?

1. Add to graylog.conf:

sidecar_collector_binary_dir = /usr/lib/custom-sidecar
sidecar_spool_dir            = /var/lib/custom-sidecar
sidecar_windows_install_dir  = C:\\Program Files\\Custom\\sidecar
sidecar_server_config_dir    = /etc/custom/server
sidecar_host_variable        = custom_host

2. Drop the already seeded data so it's re-created on server startup. In the MongoDB console:

db.sidecar_collectors.drop()
db.sidecar_configurations.drop()
db.sidecar_configuration_variables.drop()

3. Restart Graylog server, then in the UI under System -> Sidecars -> Configuration confirm:
   • opening a default configuration (e.g. filebeat-linux-default) shows /var/lib/custom-sidecar/... and ${user.custom_host}. auditbeat-linux-default shows /etc/custom/server
   • the collectors' executable paths use /usr/lib/custom-sidecar/...
   • Configuration Variables lists custom_host

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Refactoring (non-breaking change)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have requested a documentation update.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.

[Copilot]

Pull request overview

Adds new server-side configuration options to make the default Sidecar collector executable paths, spool locations, auditbeat server-config path, and the ${user.<host>} variable name configurable via graylog.conf, while keeping the existing defaults for backward compatibility.

Changes:

• Introduce new sidecar_* settings in SidecarPluginConfiguration and wire them into the default Sidecar collectors seeding migration.
• Update the default-collectors migration to substitute configurable paths and the configurable host variable name while preserving checksum-based upgrade behavior.
• Add unit tests covering configuration defaults/validation and migration behavior with custom settings.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
graylog2-server/src/test/java/org/graylog/plugins/sidecar/migrations/V20180212165000_AddDefaultCollectorsTest.java	Adds migration-level tests to verify default seeding and custom path/host-variable substitution.
graylog2-server/src/test/java/org/graylog/plugins/sidecar/common/SidecarPluginConfigurationTest.java	Adds tests for new config defaults and host-variable name validation.
graylog2-server/src/main/java/org/graylog2/commands/Server.java	Registers SidecarPluginConfiguration as a node configuration bean so the new graylog.conf options are parsed/bound.
graylog2-server/src/main/java/org/graylog/plugins/sidecar/rest/models/ConfigurationVariable.java	Extracts template-reference formatting into a reusable helper.
graylog2-server/src/main/java/org/graylog/plugins/sidecar/migrations/V20180212165000_AddDefaultCollectors.java	Uses configurable paths and host-variable name when generating default collector templates/configs.
graylog2-server/src/main/java/org/graylog/plugins/sidecar/common/SidecarPluginConfiguration.java	Adds new sidecar_* parameters (paths + host variable) and validation for host-variable naming.
changelog/unreleased/pr-26199.toml	Adds changelog entry for the new configurability feature.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[patrickmann]

This PR includes the change from #26200. What's the purpose of breaking that out into a separate PR?

[patrickmann]

1. We should document that these options set the seed-time defaults and only fully apply to freshly-seeded collectors; to change an existing deployment, drop the collections or edit collectors in the UI.
2. Warn on drift: If the configured values differ from what an existing collector carries (especially executablePath-bound ones), LOG.warn(...) that exec paths won't be auto-updated.

[xd4rker]

This PR includes the change from #26200. What's the purpose of breaking that out into a separate PR?

I kept it separate so each PR stays focused. That way, if we ever need to revert the feature, we won't have to revert the fix too, and the fix keeps its own changelog. Happy to combine them if you think that's cleaner 👍

[patrickmann]

I kept it separate so each PR stays focused

Maybe rebase 26199 on 26200 then.