fix(systemd socket activation): fix log collection for systemd sockets

[j-c-fuchs]

Summary

Systemd sockets are passed in blocking mode, but tokio expects them to be in non-blocking mode, see https://docs.rs/tokio/latest/tokio/net/struct.TcpListener.html#notes.
Therefore, always set sockets from systemd to non-blocking.

Vector configuration

# vector.yaml
sources:
  systemd_syslog_socket_activation:
    type: syslog
    #address: 127.0.0.1:12345
    address: systemd#1
    mode: udp
    decoding:
      codec: "json"

sinks:
  stdout:
    type: console
    inputs:
      - systemd_syslog_socket_activation
    encoding:
      codec: "json"
      json:
        pretty: true

How did you test this PR?

Start vector with systemd socket activation (udp socket, but tcp socket is the same), send logs and stop vector, see #24300 for more details.

#!/usr/bin/bash

send_logs() {
  sleep 5
  echo '{"hello": "world"}' | socat STDIN UDP:127.0.0.1:12345
  sleep 3
  echo '{"hello": "world again"}' | socat STDIN UDP:127.0.0.1:12345
  sleep 5
  pkill vector
  sleep 5
  echo '{"hello": "send after terminate"}' | socat STDIN UDP:127.0.0.1:12345
}

send_logs &
systemd-socket-activate \
  --datagram --listen 127.0.0.1:12345 \
  -E RUST_BACKTRACE=full \
  -E VECTOR_INTERNAL_LOG_RATE_LIMIT=0 \
  ./target/debug/vector -vvv \
  --config vector.yaml

Note: Unfortunately, I don't know how to add tests for this case. Unit tests seem impossible, because opening a socket and then using the corresponding file descriptor for listening violates io_safety. For integration tests, I don't know how to pass the file descriptor to the tests.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Closes: socket source: passed UDP file descriptor not processed until shutdown #24300

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[j-c-fuchs]

I have read the CLA Document and I hereby sign the CLA

[j-c-fuchs]

Please, let me know if I should make a changelog entry for this fix.

[j-c-fuchs]

Please, let me know if I should make a changelog entry for this fix.

Added a changelog entry.

[thomasqueirozb]

Hi, thanks for your contribution. While this may fix systemd sockets I worry it may break other system sockets that are configured differently. Have you tested this with all possible SystemFd sockets?

1. TCP blocking socket
2. TCP non-blocking socket
3. UDP blocking socket
4. UDP non-blocking socket

Maybe we could have this behavior be configured by a flag

[j-c-fuchs]

Hi Thomas,

thank you for your remarks. I tested it once again in all four configurations (without systemd) with a python script opening the sockets and then execv into vector, see the attached files. It works in all cases.

start-with-socket.py
test.sh
vector-tcp.yaml
vector-udp.yaml

Regarding the question whether to make it configurable: Blocking sockets are unsupported starting from tokio v1.44.0. Therefore, I think a flag to configure this is unnecessary and might have the danger of potential panics. See:

• Release notes for Tokio v1.44.0
• net: debug_assert on creating a tokio socket from a blocking one tokio-rs/tokio#7166
• Tokio blocking socket mitsuhiko/systemfd#34

[j-c-fuchs]

Hi @thomasqueirozb ,

have you any further questions? Can you follow my argumentation?

[aagor]

Is there something that blocks this PR? Can we do something to move forward?