Repository navigation
feat(cli): webhook notifications for autoresearch events (closes #106) - #107
Open
Srinivasan8888 wants to merge 1 commit into
Open
Srinivasan8888 wants to merge 1 commit into
Srinivasan8888 wants to merge 1 commit into
Conversation
…rd/generic) evo optimization loops run autonomously for hours, but there was no way to be told when something happened. Add an opt-in webhook notifier. - notify.py: send_notification(url, event, message, extra) POSTs a JSON payload and never raises (a down webhook must not break a run). Payload sets both `text` (Slack) and `content` (Discord) to the message plus structured fields, so one URL works for Slack, Discord, or any receiver. - `evo config set notify-webhook <url>` (http/https; empty clears; shown in `config show`). - `evo run` fires a best-effort `new_best` notification when a commit beats the previous best score. - `evo notify test` sends a test notification to verify setup. A grep of the code + docs confirms no notification/webhook integration existed anywhere; this fills that gap. Closes evo-hq#106.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #106.
Motivation
evo optimization loops run autonomously for a long time (SFT/RL cycles are 1–2h+ each, chained across many experiments), but there was no way to be notified when the loop found a new best score. Crawling the evo docs (JS-rendered) and grepping the codebase both confirm no notification/webhook/Slack integration exists anywhere — the public integrations are hosts + remote sandboxes + gates + dashboard. This fills that gap.
What's added
notify.py—send_notification(url, event, message, extra)POSTs a JSON payload and never raises (a down or misconfigured webhook must not break a run; it returns a bool). The payload sets bothtext(Slack incoming-webhook key) andcontent(Discord webhook key) to the message, plussource/event/data, so one URL works for Slack, Discord, or any generic receiver.evo config set notify-webhook <url>— stores the webhook (http/https only; empty clears; shown inconfig show, readable viaconfig get).new_bestevent — at the end ofevo run, when a commit beats the previous best (compare_scores(metric, score, best_before_score)), evo fires a best-effort notification:evo notify test— sends a test notification so users can verify setup without waiting for a run.Design notes / scope
_maybe_notifyandsend_notificationswallow all errors and only warn on stderr; a failed webhook never affects run outcome.notifymodule, the config field, thenew_bestevent, andevo notify test.Tests
tests/unit/test_notify.py(TDD, written failing first) — 11 tests:textandcontent+ structured datasend_notificationreturns True on 2xx, False on non-2xx, and False (no raise) on transport errorconfig set notify-webhookstores / rejects non-URLs / clears on empty_maybe_notifyis a no-op without a webhook and posts when one is setevo notify testposts when configured and errors clearly when notAlso verified end-to-end (
init→config set notify-webhook→config show→evo notify testfails gracefully against a bogus host, rc=1, no crash). Fulltests/unitsweep green.