Skip to content

feat: block hardcoded secrets with gitleaks at commit, push, and CI - #541

Open
arekay-nv wants to merge 2 commits into
mlcommons:mainfrom
arekay-nv:feat/gitleaks-secret-scan
Open

arekay-nv wants to merge 2 commits into
mlcommons:mainfrom
arekay-nv:feat/gitleaks-secret-scan

Conversation

@arekay-nv

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds gitleaks secret scanning at three
layers so that a hardcoded credential is stopped before it reaches main,
ideally before it leaves a contributor's machine:

Layer Where What it scans
pre-commit .pre-commit-config.yaml (gitleaks) Staged changes
pre-push .pre-commit-config.yaml (gitleaks-push) Every commit being pushed, including --no-verify commits
CI .github/workflows/secret-scan.yml Each branch push (origin/main..HEAD) and PR (base..head); full history on main

default_install_hook_types: [pre-commit, pre-push] makes a plain
pre-commit install add both hooks. Existing clones have to rerun it once
(documented in docs/DEVELOPMENT.md).

Rules (.gitleaks.toml)

  • Extends the upstream default rules (useDefault = true).
  • Adds rules for formats the defaults miss or that are specific to this project:
    passwords in DB connection URIs (postgres/mysql/mongodb/redis/amqp,
    including +driver), Neon npg_, Supabase sbp_ / sb_secret_, NVIDIA
    nvapi-, legacy NGC keys, W&B API keys, and Docker Hub dckr_pat_ /
    dckr_oat_.
  • Allowlists placeholders for those custom rules: {var}, [YOUR-PASSWORD],
    <password>, and the literal words pass / password / secret /
    changeme / redacted. These are matched against the extracted secret and
    anchored. Reviewers: check whether this set is too broad or too narrow.
  • Allowlists name=number matches for generic-api-key, which otherwise flags
    max_concurrency=100000 in docs/ENDPOINT_CLIENT.md.

.gitleaksignore holds one reviewed exception: the temporary SWE-bench service
token in examples/10_Agentic_Inference/dsv4_agentic_benchmark.yaml. With it,
a scan of main's full history is clean.

CI details

  • Runs the gitleaks CLI directly instead of gitleaks-action. The action needs
    a GITLEAKS_LICENSE secret for org repos, and secrets aren't available to
    fork PRs.
  • The binary is installed by .github/actions/install-gitleaks. It is pinned to
    the same version as the pre-commit hook (v8.30.1) and SHA256-verified.
  • The pre-commit CI jobs (GitHub and GitLab) set SKIP=gitleaks, because
    staged-change scanning has nothing to scan in CI.

Type of change

  • New feature

Testing

  • tests/unit/test_gitleaks_config.py runs the real gitleaks binary with
    the repo config. It checks that each custom rule fires on a matching
    credential, ignores placeholders, and leaves the default rules active.
    The fake credentials are generated at runtime from a fixed seed, so the
    file contains nothing push protection would flag. 23 pass. test.yml
    installs gitleaks so this file runs in CI, and it is skipped when
    gitleaks is not on PATH.
  • Checked that every custom rule fails its test before the rule is added.
  • Manually tested in a throwaway clone with a local bare remote:
    • the commit hook blocks a staged fake Neon password;
    • the pre-push hook rejects a push containing a --no-verify commit, and
      the branch never reaches the remote;
    • a clean push succeeds, and only gitleaks-push runs at push time.
  • Full-history scan of main: no findings.
  • pre-commit run --all-files: all hooks pass except the existing
    macOS-only mypy sched_setaffinity errors in unchanged source.
  • pytest -m "not slow and not performance": all gitleaks tests pass.
    The 119 failures are macOS-only (Linux CPU-affinity APIs, /proc, IPC
    socket-path length) and come from unchanged code.

Follow-ups (after merge)

  • Make Secret Scan / gitleaks a required status check on main.

Checklist

  • Code follows project style
  • Pre-commit hooks pass (macOS mypy caveat above)
  • Documentation updated (AGENTS.md, docs/DEVELOPMENT.md)

🤖 Generated with Claude Code

Add a gitleaks scan at three layers so credentials never reach main:

- pre-commit: scans staged changes.
- pre-push: rescans the commits being pushed, catching --no-verify commits.
  `default_install_hook_types` makes `pre-commit install` add both hooks.
- CI: secret-scan.yml scans every branch push and PR range with a pinned,
  checksum-verified gitleaks binary (no license key, so fork PRs work).

.gitleaks.toml extends the upstream default rules with DB-URI passwords,
Neon, Supabase, NVIDIA/NGC, W&B, and Docker Hub tokens, plus placeholder
allowlists for docs and templates. tests/unit/test_gitleaks_config.py runs
the real binary against generated fake keys to pin each rule's behavior;
test.yml installs gitleaks so it runs in CI.

.gitleaksignore records the temporary SWE-bench service token in
examples/10_Agentic_Inference/dsv4_agentic_benchmark.yaml; a full scan of
main is otherwise clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@github-actions github-actions Bot added the size/normal PR Review Policy: <=500 non-test lines & <=20 files label Oct 21, 2026
@arekay-nv
arekay-nv marked this pull request as ready for review October 22, 2026 18:10
@arekay-nv
arekay-nv requested a review from a team October 22, 2026 18:10
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@93ec321). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #541   +/-   ##
=======================================
  Coverage        ?   81.37%           
=======================================
  Files           ?      157           
  Lines           ?    22472           
  Branches        ?        0           
=======================================
  Hits            ?    18286           
  Misses          ?     4186           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/normal PR Review Policy: <=500 non-test lines & <=20 files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants