Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
9376e91
add working harbor implementation
hrdkbhatnagar Jan 13, 2026
60b8311
correctly copy metadata for judge
hrdkbhatnagar Jan 13, 2026
4a85c57
correct pass ENV vars to the verifier (harbor)fix GPU allocation bug
hrdkbhatnagar Jan 14, 2026
3447d2b
Merge remote-tracking branch 'origin/main' into add_harbor_support
hrdkbhatnagar Feb 22, 2026
0293f90
update with arenahard and healthbench, timer parity, other changes
hrdkbhatnagar Feb 24, 2026
ea7fbf3
Merge remote-tracking branch 'origin/main' into add_harbor_support
hrdkbhatnagar Apr 2, 2026
5f3cf66
Merge remote-tracking branch 'origin/main' into add_harbor_support
hrdkbhatnagar May 6, 2026
0634e7e
Fix Dockerfile to mirror opus_4_6_1m image from condor, fix timer wit…
hrdkbhatnagar May 7, 2026
52c9437
move verifier into tests/ to prevent tampering by the agent
hrdkbhatnagar May 7, 2026
22d5638
add artifact download for final_model and agent workspace
hrdkbhatnagar May 7, 2026
b31008e
use modified harbor to isolate verifier in a different sandbox
hrdkbhatnagar May 9, 2026
81b0c6e
rewrite verifer separate env with native harbor instead of using our …
hrdkbhatnagar May 15, 2026
2bc270f
Revert "rewrite verifer separate env with native harbor instead of us…
hrdkbhatnagar May 16, 2026
c9176a7
hacky:retry verifer separate env with more excludes to bypass modal i…
hrdkbhatnagar May 29, 2026
62ca4fb
use single workspace artifact instead of two; update harbor
hrdkbhatnagar Jul 8, 2026
1f82931
working end to end implementation with the PTB v1.0 setting, using mo…
hrdkbhatnagar Aug 31, 2026
68fd96b
Merge origin/main (PostTrainBench v1.1: new judges, prompt rules, inf…
hrdkbhatnagar Sep 1, 2026
2c0ca2e
v1.1 parity step 2 (prompt from get_prompt.py, info.json-driven tasks…
hrdkbhatnagar Sep 1, 2026
0236ebc
v1.1 parity step 3 (four src/judges judges in the verifier, trace han…
hrdkbhatnagar Sep 1, 2026
0c34ac5
v1.1 parity step 4 (harbor_to_results.py exporter for scripts/collect…
hrdkbhatnagar Sep 1, 2026
7b4a0eb
v1.1 parity step 5 (agent launch knobs: effort high, bash timeout, --…
hrdkbhatnagar Sep 1, 2026
6fcdf94
condor-equivalent resources, verifier agent auto-detection, codex lau…
hrdkbhatnagar Sep 1, 2026
3c71b3b
pass agent name/model to the verifier via --ve; tested end to end wit…
hrdkbhatnagar Sep 1, 2026
c0b8895
pin H100 with Modal's H100! opt-out (no H200 upgrade)
hrdkbhatnagar Sep 1, 2026
d7f5753
pin opencode/gemini CLI versions by default (harbor always npm-instal…
hrdkbhatnagar Sep 1, 2026
839d155
opencode: hand harbor PostTrainBench's opencode.json provider config …
hrdkbhatnagar Sep 1, 2026
ce33472
use upstream thinking_display kwarg (harbor#3030) for trace parity
hrdkbhatnagar Sep 1, 2026
259aceb
resolve API keys from the repo .env like the condor
hrdkbhatnagar Sep 2, 2026
216a67e
Fix --thinking-display for claude code
rank-and-file Sep 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,9 @@ __marimo__/

# testing parsed agent traces
output.txt
harbor/
harbor_old/
harbor_015/

# Nested checkout of the public website (has its own .git); never track from here
posttrainbench-website/
Expand Down
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.10
10 changes: 10 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[project]
name = "posttrainbench"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"modal>=1.3.0.post1",
"python-socks>=3.0.0",
]
377 changes: 377 additions & 0 deletions src/harbor_adapter/README.md

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions src/harbor_adapter/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
"""PostTrainBench Harbor Adapter - Generate Harbor tasks for LLM post-training evaluation."""

from .adapter import (
PostTrainBenchAdapter,
BENCHMARKS,
MODELS,
list_available_tasks,
)

__all__ = [
"PostTrainBenchAdapter",
"BENCHMARKS",
"MODELS",
"list_available_tasks",
]
Loading