scripts: add consensus log downloader config infrastructure and args parsing

[lev-starkware]

Note

Low Risk
Adds a new standalone script and ignore rules without wiring it into production code paths. Main risk is minor: argument/config validation behavior and dependency on env_map.yaml/PyYAML for local tooling.

Overview
Introduces a new scripts/consensus_report scaffold for generating consensus log reports.

Adds generate_logs_and_report.py with env_map.yaml loading/validation (via yaml.safe_load) and a CLI that accepts environment selection plus block height and time-window/report output options; core log download/report generation is left as TODOs. Also adds a local .gitignore to prevent committing env_map.yaml and generated *.json/*.txt artifacts.

^{Written by Cursor Bugbot for commit b718a83. This will update automatically on new commits. Configure here.}

[reviewable-StarkWare]

This change is 

[lev-starkware]

• scripts: add to change consensus report --last HH:MM instead of --last-24-hours #13616
• scripts: implement generate report #13209
• scripts: consensus report rendering pt2 — tables, evidence, orchestration, main #13208
• scripts: consensus report rendering pt1 — nodes section, round metadata, per-node row #13206
• scripts: consensus report data pipeline — load, index, extract votes and validation #13205
• scripts: consensus report domain helpers — ntxs, weights, closing reason, adv step #13204
• scripts: consensus report domain helpers — round timing and proposer resolution #13198
• scripts: consensus report ASCII table rendering #13197
• scripts: consensus report log-parsing utilities and ROUND_RE #13195
• scripts: consensus report scaffold — imports, dataclasses, stub main #13186
• scripts: add log downloading with report generation stub #12842
• scripts: add filter preparation and time window computation #12841
• scripts: begining of the log filter preparation #12840
• scripts: add consensus log downloader config infrastructure and args parsing #12839 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[lev-starkware]

@lev-starkware made 3 comments and resolved 4 discussions.
Reviewable status: 0 of 2 files reviewed, all discussions resolved (waiting on TzahiTaub).

[TzahiTaub]

@TzahiTaub reviewed 2 files and all commit messages, and made 11 comments.
Reviewable status: all files reviewed, 11 unresolved discussions (waiting on lev-starkware).

---

scripts/consensus_report/.gitignore line 1 at r4 (raw file):

env_map.yaml

Suggestion:

env_map.yaml  # Should be supplied by the user and never committed.

---

scripts/consensus_report/.gitignore line 7 at r4 (raw file):

*.pyc
.venv/
venv/

This is needed if you plan to create a venv here as a part of the script. If you do, pick one of the names, no need for both. If you don't, delete both.

Code quote:

.venv/
venv/

---

scripts/consensus_report/generate_logs_and_report.py line 16 at r4 (raw file):

class EnvConfig:
    project: str
    namespace_re: str

Suggestion:

namespace_regex

---

scripts/consensus_report/generate_logs_and_report.py line 36 at r4 (raw file):

    environments = data.get("environments", {})
    default_env = data.get("default_environment", "integration")

I think it's better to remove this and let the user always state the env explicitly.

---

scripts/consensus_report/generate_logs_and_report.py line 57 at r4 (raw file):

    ap.add_argument("--height", required=True, type=int, help="Block height, e.g. 6591090")
    ap.add_argument(
        "--out_json",

Suggestion:

   "--out_json_path",

---

scripts/consensus_report/generate_logs_and_report.py line 67 at r4 (raw file):

    ap.add_argument("--start", help="RFC3339 timestamp - time window start (requires --end)")
    ap.add_argument("--end", help="RFC3339 timestamp - time window end (requires --start)")
    ap.add_argument("--near", help="RFC3339 timestamp - search near this time (±2h window)")

Give the format and an example here instead of the RFC to avoid an additional check of what RFC3339 is.

• I prefer to have the simplest to use, good enough format, this one seems too detailed for our needs. Wrote in the diff what I would expect. Same for the other two.

Suggestion:

    ap.add_argument("--near", help="YY-MM-DD HH::MM formated timestamp- search near this time (±2h window)")

---

scripts/consensus_report/generate_logs_and_report.py line 71 at r4 (raw file):

        "--today",
        action="store_true",
        help="Use today's UTC midnight-to-midnight time window (default if no time args)",

What does UTC here means? I we run this script at local time (utc+2) of 01:59 == 23:59 utc will we get the last two hours data (this is what we want) or the data of yesterday?

Code quote:

 help="Use today's UTC midnight-to-midnight time window (default if no time args)",

---

scripts/consensus_report/generate_logs_and_report.py line 72 at r4 (raw file):

        action="store_true",
        help="Use today's UTC midnight-to-midnight time window (default if no time args)",
    )

I think last 2 hours (or some other time) is better - this will probably be the used option when debugging in real time, and if we look at a consensus height that is around midnight, the "today" will give partial data.
As a note - let the script error if the logs we retrieve miss the start log of the height, and print a bold errorif we miss the next height start log - so that a user will now it get partial data. (I think it shouldn't error in the latter as we might want to run this during a height that is stuck).

Code quote:

    ap.add_argument(
        "--today",
        action="store_true",
        help="Use today's UTC midnight-to-midnight time window (default if no time args)",
    )

---

scripts/consensus_report/generate_logs_and_report.py line 79 at r4 (raw file):

    )
    ap.add_argument(
        "--report",

Suggestion:

 "--out_report_path",

---

scripts/consensus_report/generate_logs_and_report.py line 80 at r4 (raw file):

    ap.add_argument(
        "--report",
        help="Generate report to this file (requires --out_json). Extension .txt added if missing.",

I think it's more convenient to have the option to avoid the--out_json ' flag if we call --report. Have the out_json` the same as the report name (with the diff in extension) if not provided explicitly.

Suggestion:

      help="Report file path to generate. Extension .txt is added if missing.",

---

scripts/consensus_report/generate_logs_and_report.py line 71 at r4 (raw file):

        "--today",
        action="store_true",
        help="Use today's UTC midnight-to-midnight time window (default if no time args)",

Add assertions for the input - e.g, start and end come together, one of this group or auto must provided.

Code quote:

    ap.add_argument("--start", help="RFC3339 timestamp - time window start (requires --end)")
    ap.add_argument("--end", help="RFC3339 timestamp - time window end (requires --start)")
    ap.add_argument("--near", help="RFC3339 timestamp - search near this time (±2h window)")
    ap.add_argument(
        "--today",
        action="store_true",
        help="Use today's UTC midnight-to-midnight time window (default if no time args)",

[lev-starkware]

@lev-starkware made 10 comments and resolved 1 discussion.
Reviewable status: all files reviewed, 10 unresolved discussions (waiting on TzahiTaub).

---

scripts/consensus_report/.gitignore line 7 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

This is needed if you plan to create a venv here as a part of the script. If you do, pick one of the names, no need for both. If you don't, delete both.

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 36 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

I think it's better to remove this and let the user always state the env explicitly.

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 67 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

Give the format and an example here instead of the RFC to avoid an additional check of what RFC3339 is.

• I prefer to have the simplest to use, good enough format, this one seems too detailed for our needs. Wrote in the diff what I would expect. Same for the other two.

Done. As discussed

---

scripts/consensus_report/generate_logs_and_report.py line 71 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

Add assertions for the input - e.g, start and end come together, one of this group or auto must provided.

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 71 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

What does UTC here means? I we run this script at local time (utc+2) of 01:59 == 23:59 utc will we get the last two hours data (this is what we want) or the data of yesterday?

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 72 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

I think last 2 hours (or some other time) is better - this will probably be the used option when debugging in real time, and if we look at a consensus height that is around midnight, the "today" will give partial data.
As a note - let the script error if the logs we retrieve miss the start log of the height, and print a bold errorif we miss the next height start log - so that a user will now it get partial data. (I think it shouldn't error in the latter as we might want to run this during a height that is stuck).

Done. changed to last-24-hours

---

scripts/consensus_report/generate_logs_and_report.py line 80 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

I think it's more convenient to have the option to avoid the--out_json ' flag if we call --report. Have the out_json` the same as the report name (with the diff in extension) if not provided explicitly.

Done. Already done, only forgot to change a message.

---

scripts/consensus_report/.gitignore line 1 at r4 (raw file):

env_map.yaml

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 57 at r4 (raw file):

    ap.add_argument("--height", required=True, type=int, help="Block height, e.g. 6591090")
    ap.add_argument(
        "--out_json",

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 79 at r4 (raw file):

    )
    ap.add_argument(
        "--report",

Done. Agreed on --report_path

[TzahiTaub]

@TzahiTaub reviewed 1 file and all commit messages, made 4 comments, and resolved 8 discussions.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on lev-starkware).

---

scripts/consensus_report/.gitignore line 1 at r4 (raw file):

Previously, lev-starkware wrote…

Done.

Not pushed

---

scripts/consensus_report/.gitignore line 7 at r4 (raw file):

Previously, lev-starkware wrote…

Done.

Not pushed

---

scripts/consensus_report/generate_logs_and_report.py line 71 at r5 (raw file):

    ap.add_argument(
        "--start",
        help="TIMESTAMP - time window start (requires --end). TIMESTAMP format: YYYY-MM-DDTHH:MM:SSZ",

The Z stands for UTC right? We want to use local time. Please add to the help message as well - "in local time".

Code quote:

TIMESTAMP format: YYYY-MM-DDTHH:MM:SSZ",

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[lev-starkware]

@lev-starkware made 4 comments and resolved 3 discussions.
Reviewable status: 0 of 2 files reviewed, 3 unresolved discussions (waiting on TzahiTaub).

---

scripts/consensus_report/.gitignore line 1 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

Not pushed

Done.

---

scripts/consensus_report/.gitignore line 7 at r4 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

Not pushed

Done.

---

scripts/consensus_report/generate_logs_and_report.py line 71 at r4 (raw file):

Previously, lev-starkware wrote…

Done.

In following PRs.

---

scripts/consensus_report/generate_logs_and_report.py line 71 at r5 (raw file):

Previously, TzahiTaub (Tzahi) wrote…

The Z stands for UTC right? We want to use local time. Please add to the help message as well - "in local time".

Done.

[TzahiTaub]

@TzahiTaub reviewed 2 files and all commit messages, made 1 comment, and resolved 3 discussions.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on lev-starkware).