Here’s a clean, product-level ticket you can drop into GitHub / Linear / Jira:
⸻
Title
GitHub Actions Integration: Validate LLM API Keys at Runtime Using KeyProbe
⸻
Description
Add support and documentation for using KeyProbe inside GitHub Actions to validate LLM API keys during CI/CD.
The goal is to fail builds early if a provided API key is invalid or lacks required capabilities.
This enables teams to:
• Verify secrets injected via GitHub Actions (secrets.*)
• Prevent deployments with broken or misconfigured LLM credentials
• Ensure required capabilities (chat, streaming, model access, etc.) are available before runtime
⸻
Problem
Currently, API key issues are discovered late:
• During runtime
• After deployment
• Through vague API errors
This leads to:
• Failed jobs in production
• Debugging overhead
• Unclear root cause (invalid key vs missing capability)
⸻
Proposed Solution
Enable a CI step using KeyProbe that:
1. Reads API key from GitHub Actions secrets
2. Runs KeyProbe against the key
3. Fails the pipeline if:
• Key is invalid (exit code != 0)
• Required capabilities are missing (future enhancement via JSON parsing)
⸻
Example Use Case
A team deploying an LLM-powered service wants to ensure:
• The API key is valid
• The key has access to specific models (e.g., Sonnet, GPT-4 class)
• Streaming is available
Before allowing the build/release step to proceed.
⸻
Scope
• Document GitHub Actions usage
• Ensure CLI exit codes are reliable for CI usage
• Support JSON output parsing for capability checks
• Provide example workflow snippet in README
⸻
Acceptance Criteria
• KeyProbe works seamlessly in GitHub Actions environment
• Build fails when invalid API key is provided
• Documentation includes GitHub Actions example
• JSON output can be used to assert capabilities (optional enhancement)
• Exit codes are clearly defined and consistent
⸻
Future Enhancements
• Add --require flag (e.g. --require streaming)
• Add --require-model support
• Native GitHub Action wrapper (composite action)
• Support multiple keys/providers in a single run
⸻
Notes
This turns KeyProbe from a debugging tool into a preventive layer in CI/CD, especially useful for teams working with multiple LLM providers.
⸻
If you want, I can also  give you a ready-to-paste GitHub Actions workflow next.
Here’s a clean, product-level ticket you can drop into GitHub / Linear / Jira:
⸻
Title
GitHub Actions Integration: Validate LLM API Keys at Runtime Using KeyProbe
⸻
Description
Add support and documentation for using KeyProbe inside GitHub Actions to validate LLM API keys during CI/CD.
The goal is to fail builds early if a provided API key is invalid or lacks required capabilities.
This enables teams to:
• Verify secrets injected via GitHub Actions (secrets.*)
• Prevent deployments with broken or misconfigured LLM credentials
• Ensure required capabilities (chat, streaming, model access, etc.) are available before runtime
⸻
Problem
Currently, API key issues are discovered late:
• During runtime
• After deployment
• Through vague API errors
This leads to:
• Failed jobs in production
• Debugging overhead
• Unclear root cause (invalid key vs missing capability)
⸻
Proposed Solution
Enable a CI step using KeyProbe that:
1. Reads API key from GitHub Actions secrets
2. Runs KeyProbe against the key
3. Fails the pipeline if:
• Key is invalid (exit code != 0)
• Required capabilities are missing (future enhancement via JSON parsing)
⸻
Example Use Case
A team deploying an LLM-powered service wants to ensure:
• The API key is valid
• The key has access to specific models (e.g., Sonnet, GPT-4 class)
• Streaming is available
Before allowing the build/release step to proceed.
⸻
Scope
• Document GitHub Actions usage
• Ensure CLI exit codes are reliable for CI usage
• Support JSON output parsing for capability checks
• Provide example workflow snippet in README
⸻
Acceptance Criteria
• KeyProbe works seamlessly in GitHub Actions environment
• Build fails when invalid API key is provided
• Documentation includes GitHub Actions example
• JSON output can be used to assert capabilities (optional enhancement)
• Exit codes are clearly defined and consistent
⸻
Future Enhancements
• Add --require flag (e.g. --require streaming)
• Add --require-model support
• Native GitHub Action wrapper (composite action)
• Support multiple keys/providers in a single run
⸻
Notes
This turns KeyProbe from a debugging tool into a preventive layer in CI/CD, especially useful for teams working with multiple LLM providers.
⸻
If you want, I can also  give you a ready-to-paste GitHub Actions workflow next.