feat(evaluators): add defenseclaw evaluator

[namrataghadi-galileo]

Summary

Added two global external evaluator integrations:

• defenseclaw.rule_pack

• defenseclaw.opa_policy

• Added strict, typed Pydantic configuration models and distinct JSON Schemas.

• Added dedicated Agent Control UI forms with canonical serialization of hidden protocol fields.

• Both evaluator execution paths are intentionally no-ops and do not install or invoke DefenseClaw OSS packages.

Scope

User-facing/API changes:

• Both evaluators appear independently in GET /api/v1/evaluators.
• Added typed UI forms for rule-pack and OPA-policy configuration.
• Rule-pack forms support adding and removing rules.
• Fixed protocol fields remain hidden in Form mode but are always serialized.
• DefenseClaw configurations are preserved in get_server_controls() output.

Internal changes:

• Added the agent-control-evaluator-defenseclaw contrib package.
• Added evaluator entry-point registration and package/release wiring.
• Added strict nested validation, including unique rule IDs and confidence bounds.
• Added default configurations and UI registry entries.
• Added focused backend and UI tests.

Out of scope:

• DefenseClaw transport, authentication, or provider invocation.
• Installing or calling DefenseClaw OSS packages.
• Changes to the separate ~/code/ui Galileo UI repository.
• Orbit production packaging changes.

Risk and Rollout

Risk level: medium

Primary risk:

• The evaluator package must be installed in the server runtime for catalog discovery and server execution.
• It must also be installed in SDK runtimes for controls using execution: "sdk".

Rollout requirements:

• Publish agent-control-evaluator-defenseclaw.
• Add the package to the Orbit Agent Control image dependencies.
• Verify both evaluator identifiers appear in GET /api/v1/evaluators before activating controls.

Rollback plan:

• Remove or disable controls referencing the DefenseClaw evaluator names.
• Remove the DefenseClaw package from runtime dependencies.
• Revert the evaluator registry and UI form registration changes.

Testing

• Added or updated automated tests

  • Ran make check (full repository check was not run).
  • Ran 26 focused Python tests successfully.
  • Ran Ruff and mypy successfully.
  • Ran TypeScript, ESLint, and Prettier checks successfully.
  • Verified contrib package wiring, evaluator registration, and server catalog schemas.
  • Playwright could not run in the sandbox because it could not bind the local UI server port.

• Manually verified behavior

  • Registration and generated catalog schemas were verified programmatically.
  • Full browser-based Form/JSON interaction should be verified locally using the supplied testing guide.

Checklist

• Linked issue/spec

  • Technical spec: defenseclaw-external-evaluators-technical-spec.md
  • Add the appropriate issue/spec URL before merging.

• Updated docs/examples for user-facing changes

  • Added the contrib package README.
  • Created a first-time local setup and testing guide.

• Included required follow-up tasks

  • Add DefenseClaw packaging to ~/code/orbit.
  • Implement the corresponding forms in ~/code/ui.
  • Run the full Playwright suite.
  • Install the evaluator package in every SDK runtime that evaluates DefenseClaw controls.

[codecov]

Codecov Report

❌ Patch coverage is 94.17476% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...rc/agent_control_evaluator_defenseclaw/__init__.py	77.77%	2 Missing ⚠️
...trol_evaluator_defenseclaw/opa_policy/evaluator.py	88.88%	2 Missing ⚠️
...ntrol_evaluator_defenseclaw/rule_pack/evaluator.py	88.88%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!