Skip to content

[P1][D2] Align docs to reality: audit ARCHITECTURE.md, mark Implemented/Partial/Planned #14

Description

@POWDER-RANGER

Problem

docs/ARCHITECTURE.md contains security and capability claims that don't match the current implementation. This directly damages credibility with contributors and security auditors — and is specifically flagged by OpenSSF Scorecard reviewers who read documentation.

Known gaps (documented in ENHANCEMENT_ROADMAP.md itself):

Claim in ARCHITECTURE.md Actual Status
AES-256-GCM vault encryption base64 stub (until issue #8 merges)
PDDL-based symbolic planning greedy forward search (not PDDL)
Multi-agent task distribution single-agent execute_task() only
Audit log export partial/unverified
Governance policy enforcement present but untested

Tasks

Audit Phase

  • Read every claim in docs/ARCHITECTURE.md and cross-reference with actual source code
  • Build a feature status matrix: Implemented / Partial / Planned
  • Read every claim in README.md and verify accuracy
  • Identify all broken internal doc links
  • Check if docs/ links referenced in README actually exist

Documentation Status Matrix (add to ARCHITECTURE.md)

## Implementation Status

> Last updated: 2026-02-25  
> Legend: ✅ Implemented | 🔶 Partial | 🔲 Planned

| Component | Feature | Status | Notes |
|-----------|---------|--------|-------|
| Vault | Key-value store API | ✅ Implemented | store/retrieve/delete/list |
| Vault | AES-256-GCM encryption | 🔶 Partial | PR #8 in progress |
| Vault | Key derivation (PBKDF2) | 🔲 Planned | Issue #8 |
| Vault | Key rotation | 🔲 Planned | Issue #8 |
| Vault | Persistent storage | ✅ Implemented | JSON file backend |
| Agents | Agent lifecycle (start/stop) | ✅ Implemented | |
| Agents | Task execution | ✅ Implemented | execute_task() |
| Agents | Task queue (assign/wait) | 🔲 Planned | Issue #7 |
| Agents | Multi-agent coordination | 🔶 Partial | AgentManager exists |
| Core | GovernanceEngine | 🔶 Partial | Policy eval present |
| Core | Governance audit log | 🔶 Partial | Needs verification |
| Core | SymbolicPlanner | ✅ Implemented | Greedy forward search |
| Core | PDDL integration | 🔲 Planned | Not PDDL — custom search |
| CLI | oblisk_cli.py | ✅ Implemented | Functional |
| CLI | Interactive TUI | 🔶 Partial | Basic only |
| Tests | Unit test suite | 🔲 Planned | Issue #9 |
| Tests | 100% coverage | 🔲 Planned | Issues #9, #10 |
| CI | Real pytest pipeline | 🔲 Planned | Issue #10 |
| Security | CodeQL SAST | 🔲 Planned | Issue #12 |
| Security | Signed releases | 🔲 Planned | Issue #13 |

Fix ARCHITECTURE.md

  • Add status matrix table (above) to top of ARCHITECTURE.md
  • Change "AES-256-GCM encryption" heading to "Vault Encryption (see status matrix)"
  • Add note: "Symbolic Planner uses greedy forward search, not PDDL. PDDL integration is planned."
  • Fix any broken Mermaid diagrams (if present)
  • Add "Architecture Accuracy" notice at top:
    > ⚠️ **Accuracy Notice:** Architecture descriptions reflect the design vision.
    > See the Implementation Status matrix below for current feature completeness.
    > Claims marked 🔲 Planned are aspirational and not yet implemented.

Fix README.md

  • Add prominent Project Status badge section near top:
    ## Project Status
    
    ![CI](https://github.com/POWDER-RANGER/OBLISK/actions/workflows/ci.yml/badge.svg)
    [![Coverage](https://codecov.io/gh/POWDER-RANGER/OBLISK/branch/main/graph/badge.svg)](https://codecov.io/gh/POWDER-RANGER/OBLISK)
    [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/POWDER-RANGER/OBLISK/badge)](https://securityscorecards.dev/viewer/?uri=github.com/POWDER-RANGER/OBLISK)
    
    **Status:** Alpha — core framework functional, encryption hardening in progress.
  • Replace any claim of "AES-256-GCM" with "AES-256-GCM (implementing — see roadmap)" until [P0][A3] Replace base64 stub with real AES-256-GCM encryption in Vault #8 merges
  • Add disclaimer: "Features listed reflect the roadmap; see docs/ARCHITECTURE.md for implementation status."
  • Verify all Quick Start code examples actually run

Fix SECURITY.md

  • Add vulnerability reporting timeline (respond within 48h, patch within 14 days)
  • Document supported versions table
  • Add disclosure policy (coordinated disclosure, 90-day window)
  • Add security contact email or GitHub private advisory link
  • Document which cryptographic primitives are used (updated once [P0][A3] Replace base64 stub with real AES-256-GCM encryption in Vault #8 merges)

Docs Directory Audit

  • List all files in docs/ — verify each is linked and current
  • Create docs/SECURITY-MODEL.md explaining vault trust boundaries
  • Create docs/CONTRIBUTING-SECURITY.md for security-focused contributions
  • Ensure no doc file claims a feature without cross-referencing the status matrix

Continuous Accuracy

  • Add docs accuracy as a required PR checklist item in .github/PULL_REQUEST_TEMPLATE.md:
    ## Documentation
    - [ ] ARCHITECTURE.md status matrix updated if feature status changed
    - [ ] README claims match implemented code
    - [ ] New features marked 🔲 Planned until tests pass

Acceptance Criteria

  • docs/ARCHITECTURE.md contains status matrix with accurate Implemented/Partial/Planned labels
  • ✅ Vault section no longer claims AES-256-GCM is fully implemented (until [P0][A3] Replace base64 stub with real AES-256-GCM encryption in Vault #8 merges)
  • ✅ Planner section accurately describes greedy forward search (not PDDL)
  • ✅ README has status badges for CI, coverage, and Scorecard
  • ✅ README does not claim any unimplemented feature as complete
  • ✅ SECURITY.md has vulnerability reporting timeline and supported versions
  • ✅ PR template requires docs accuracy check
  • ✅ All internal links in docs/ and README.md resolve without 404

Why This Matters Beyond Trust

OpenSSF Scorecard's Maintained check looks at activity and responsiveness. Accurate docs demonstrate project maturity. Security auditors who find documentation mismatches will rate the project as untrustworthy regardless of code quality. Since OBLISK is positioned as a security framework, documentation accuracy is a security property, not just a UX concern.

Dependencies

Requires: Nothing (doc edits are safe to merge any time)
Should be updated when: #7, #8, #9, #10, #12, #13 merge
Estimated effort: 3-4 hours initial + 30 min per feature PR

Priority: P1
Labels: documentation, security, accuracy, P1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions