Add date_diff pipeline function for date difference calculations

[danotorrey]

What

Adds a new date_diff pipeline function. Fixes #26142.

Function shape

date_diff(left: DateTime, right: DateTime, absolute: boolean = false) -> Map

arg	type	required	meaning
left	DateTime	yes	Start of the interval
right	DateTime	yes	End of the interval
absolute	boolean	no	If true, return absolute numeric values. Default false.

left may be before or after right — the result is signed by default. Inputs must be DateTime — feed strings through parse_date (or flex_parse_date) first.

Result shape

A map keyed by unit (every unit expresses the full interval rounded to the nearest whole unit — drop to a smaller unit if you need more precision), plus direction and friendly:

{
  millis:    <long>,
  seconds:   <long>,
  minutes:   <long>,
  hours:     <long>,
  days:      <long>,
  weeks:     <long>,
  direction: "ahead" | "behind" | "equal",
  friendly:  <string>
}

E.g. a 2-day interval (end after start) yields:

{
  millis:    172800000,
  seconds:   172800,
  minutes:   2880,
  hours:     48,
  days:      2,
  weeks:     0,
  direction: "ahead",
  friendly:  "2 days"
}

• Numeric values are signed by default (end − start). Pass absolute: true for absolute values.
• direction describes the end relative to the start: "ahead" (later), "behind" (earlier), "equal" (same instant). Always derived from the signed result, so it is preserved even when absolute: true.
• friendly is a human-readable rendering. Zero-valued components are omitted. For intervals shorter than one minute, sub-second remainder is included (e.g. "250 ms", "1 second 500 ms"); for longer intervals the millisecond remainder is suppressed to avoid noise — the raw millis field always has the exact value. Negative intervals are prefixed with -. Further examples: "30 seconds", "2 days", "-2 days", "1 week 1 day 3 hours 15 minutes".

Months and years are deliberately omitted because they aren't constant-length; a calendar-aware helper can land separately if there's demand.

Why

There was no first-class way to compute date differences in a pipeline rule. The only path was dateA - dateB (returns a Joda Duration) followed by to_string(...) and a regex against the ISO‑8601 form (PT123S vs PT123.456S) to recover a numeric value — brittle and easy to get wrong.

Examples

All examples below are exercised end-to-end by FunctionsSnippetsTest#dateDiffPrExamples so they're guaranteed to parse and execute.

1. VPN session duration from a RADIUS accounting record

Input message has acct_session_start like "2025-05-27T13:42:10.000+0000" and the message timestamp is when accounting-stop hit the pipeline.

rule "vpn session duration"
when
    has_field("acct_session_start")
then
    let start_dt = parse_date(value: to_string($message.acct_session_start),
                              pattern: "yyyy-MM-dd'T'HH:mm:ss.SSSZ");
    let end_dt   = to_date($message.timestamp);

    let session = date_diff(start_dt, end_dt);
    set_field("session_seconds", session.seconds);
    set_field("session_minutes", session.minutes);
    set_field("session_hours",   session.hours);
    set_field("session_display", session.friendly);   // e.g. "17 minutes 50 seconds"
end

2. Account age at login (flag brand-new accounts)

Auth event has user_created like "03/15/2024".

rule "tag new account logins"
when
    has_field("event_type") && to_string($message.event_type) == "user_login"
then
    let created = parse_date(value: to_string($message.user_created),
                             pattern: "MM/dd/yyyy");
    let age = date_diff(left: created, right: now(), absolute: true);

    set_field("account_age_days", age.days);
    set_field("account_is_new",   to_long(age.days) < 7);
end

Note: map field access (age.days) returns Object, so it needs to_long(...) before numeric comparison.

3. HTTP request latency from access log fields

Access log has request_received_at and response_sent_at as ISO timestamps.

rule "http latency"
when
    has_field("request_received_at") && has_field("response_sent_at")
then
    let req = parse_date(value: to_string($message.request_received_at),
                         pattern: "yyyy-MM-dd'T'HH:mm:ss.SSSZ");
    let res = parse_date(value: to_string($message.response_sent_at),
                         pattern: "yyyy-MM-dd'T'HH:mm:ss.SSSZ");

    let latency = date_diff(req, res);
    set_field("latency_ms",      latency.millis);
    set_field("latency_seconds", latency.seconds);
end

4. Stale record detection (uses friendly and direction)

rule "stale record age"
when
    has_field("last_updated")
then
    let updated = parse_date(value: to_string($message.last_updated), pattern: "MM/dd/yyyy");
    let age = date_diff(updated, now(), absolute: true);

    set_field("age_display", age.friendly);             // e.g. "3 days 4 hours"
    set_field("is_stale",    to_long(age.days) > 30);
    set_field("direction",   age.direction);            // "ahead" / "behind" / "equal"
end

Test plan

• ./mvnw -pl :graylog2-server -Dtest='FunctionsSnippetsTest#dateDiff+dateDiffPrExamples' test -Dskip.web.build=true -Dmaven.javadoc.skip=true
• Manual: create a rule using date_diff against a real message, confirm each unit field matches expectations.

Assisted with Claude Code

[kingzacko1]

Should we really do half-away-from-zero rounding? Seems like if we want to keep whole numbers then truncation may be the better option. If you were to add a days field in your pipeline processing and then query for days < 7 you're really getting days < 6.5. @danotorrey

[kingzacko1]

@danotorrey Everything with the exception of the rounding logic LGTM. Not sure if its just what Claude defaulted to, but a simple truncated rounding makes more sense to me than half-from-zero unless we have some other reason to go with it.