scripts: add filter preparation and time window computation

[lev-starkware]

Note

Medium Risk
Adds new time-window selection logic that shells out to gcloud logging read and performs timezone/RFC3339 parsing; incorrect parsing or missing markers could lead to wrong log ranges or script failures.

Overview
Adds end-to-end time window computation to generate_logs_and_report.py and wires it into prepare_filter to return the final bounded log filter plus start_time/end_time.

Introduces --auto windowing by querying the first Running consensus for height N (and optionally N+1) timestamp via gcloud logging read, applying a default 15m end and ±30s buffer when needed. Also adds --near (±2h) and explicit --start/--end handling using local Asia/Jerusalem timestamps converted to UTC, along with argument conflict validation and a --print-filters debug path.

^{Written by Cursor Bugbot for commit 393bf5c. This will update automatically on new commits. Configure here.}

[reviewable-StarkWare]

This change is 

[lev-starkware]

• scripts: implement generate report #13209
• scripts: consensus report rendering pt2 — tables, evidence, orchestration, main #13208
• scripts: consensus report rendering pt1 — nodes section, round metadata, per-node row #13206
• scripts: consensus report data pipeline — load, index, extract votes and validation #13205
• scripts: consensus report domain helpers — ntxs, weights, closing reason, adv step #13204
• scripts: consensus report domain helpers — round timing and proposer resolution #13198
• scripts: consensus report ASCII table rendering #13197
• scripts: consensus report log-parsing utilities and ROUND_RE #13195
• scripts: consensus report scaffold — imports, dataclasses, stub main #13186
• scripts: add log downloading with report generation stub #12842
• scripts: add filter preparation and time window computation #12841 👈 (View in Graphite)
• scripts: begining of the log filter preparation #12840
• scripts: add consensus log downloader config infrastructure and args parsing #12839
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[lev-starkware]

@lev-starkware made 1 comment and resolved 1 discussion.
Reviewable status: 0 of 1 files reviewed, all discussions resolved (waiting on TzahiTaub).

[lev-starkware]

@lev-starkware made 1 comment and resolved 1 discussion.
Reviewable status: 0 of 1 files reviewed, all discussions resolved (waiting on TzahiTaub).

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[lev-starkware]

@lev-starkware resolved 1 discussion.
Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on TzahiTaub).

[TzahiTaub]

@TzahiTaub made 7 comments.
Reviewable status: 0 of 1 files reviewed, 7 unresolved discussions (waiting on lev-starkware).

---

scripts/consensus_report/generate_logs_and_report.py line 182 at r7 (raw file):

def first_timestamp(project: str, flt: str) -> str:

To hint a remote query.

Suggestion:

def retrieve_first_timestamp(

---

scripts/consensus_report/generate_logs_and_report.py line 77 at r7 (raw file):

    dt_naive = datetime.fromisoformat(ts)
    israel_tz = ZoneInfo("Asia/Jerusalem")
    dt_ist = dt_naive.replace(tzinfo=israel_tz)

The script should be runnable in every time zone. Calling astimezone on a non aware datetime treats the original object as in local time.

Suggestion:

    dt_without_tz = datetime.fromisoformat(ts)
    dt_with_tz = dt_naive.astimezone()

---

scripts/consensus_report/generate_logs_and_report.py line 108 at r7 (raw file):

    Convert IST (Israel Standard Time) datetime to UTC.
    """
    return ist_time.astimezone(timezone.utc)

Function works for every timezone, and the script shouldn't limit usage to a specific timezone.

Suggestion:

def to_utc(local_time: datetime) -> datetime:
    """
    Converts local datetime to UTC.
    """
    return local_time.astimezone(timezone.utc)

---

scripts/consensus_report/generate_logs_and_report.py line 217 at r7 (raw file):

        raise RuntimeError(
            "--auto, --near, --start/--end, and --last-24-hours are mutually exclusive"
        )

Consider removing and using add_mutually_exclusive_group as suggested below

Code quote:

    # Check for conflicts between time options
    options = [args.auto, args.near, (args.start or args.end), args.last_24_hours]
    if sum(bool(o) for o in options) > 1:
        raise RuntimeError(
            "--auto, --near, --start/--end, and --last-24-hours are mutually exclusive"
        )

---

scripts/consensus_report/generate_logs_and_report.py line 223 at r7 (raw file):

            raise RuntimeError("--auto requires environment config")
        start_ts = first_timestamp(
            environment.project, consensus_height_filter(common_filter_prefix, args.height)

As this will always be called with no optional args, we can simplify the code a bit.

Suggestion:

def determine_search_window(
    args: argparse.Namespace,
    environment: EnvConfig,
    common_filter_prefix: str,
) -> Tuple[datetime, datetime]:
    """Determine the search time window based on provided arguments.

    Priority/validation:
      - --auto, --near, --start/--end, --last-24-hours are mutually exclusive
      - --start/--end must be provided together
      - --last-24-hours (or no args) uses (current_time - 24 hours) to current_time window
      - --auto requires environment and common_filter_prefix parameters
    """

    # Check for conflicts between time options
    options = [args.auto, args.near, (args.start or args.end), args.last_24_hours]
    if sum(bool(o) for o in options) > 1:
        raise RuntimeError(
            "--auto, --near, --start/--end, and --last-24-hours are mutually exclusive"
        )

    if args.auto:
        start_ts = first_timestamp(
            environment.project, consensus_height_filter(common_filter_prefix, args.height)

---

scripts/consensus_report/generate_logs_and_report.py line 227 at r7 (raw file):

        if not start_ts:
            raise RuntimeError(
                f"START_MARKER not found: Running consensus for height {args.height}"

Use the const you defined

Code quote:

Running consensus for height

---

scripts/consensus_report/generate_logs_and_report.py line 305 at r7 (raw file):

        action="store_true",
        help=f"Auto-detect time window by searching for '{RUNNING_CONSENSUS_FOR_HEIGHT} N' markers",
    )

Consider using the builtin argparse option for mutuallu exclusive flags

Suggestion:

    time_group = ap.add_mutually_exclusive_group()
    time_group.add_argument(
        "--auto",
        action="store_true",
        help=f"Auto-detect time window by searching for '{RUNNING_CONSENSUS_FOR_HEIGHT} N' markers",
    )

[lev-starkware]

@lev-starkware made 6 comments and resolved 3 discussions.
Reviewable status: 0 of 1 files reviewed, 4 unresolved discussions (waiting on TzahiTaub).

---

scripts/consensus_report/generate_logs_and_report.py line 77 at r7 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

The script should be runnable in every time zone. Calling astimezone on a non aware datetime treats the original object as in local time.

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 108 at r7 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

Function works for every timezone, and the script shouldn't limit usage to a specific timezone.

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 182 at r7 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

To hint a remote query.

Done

---

scripts/consensus_report/generate_logs_and_report.py line 223 at r7 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

As this will always be called with no optional args, we can simplify the code a bit.

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 227 at r7 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

Use the const you defined

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 305 at r7 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

Consider using the builtin argparse option for mutuallu exclusive flags

Done + changed --start/--end to --range START END

[TzahiTaub]

@TzahiTaub reviewed 1 file and all commit messages, and resolved 4 discussions.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on lev-starkware).