Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
6383d4f
feat(stt): add Modulate Velma-2 streaming STT provider (#7140)
beastoin May 3, 2026
17f9f05
feat(stt): add Modulate Velma-2 batch pre-recorded transcription (#7140)
beastoin May 3, 2026
04051ca
feat(stt): wire Modulate socket into streaming handler (#7140)
beastoin May 3, 2026
e4dcf7a
test(stt): add 36 unit tests for Modulate STT integration (#7140)
beastoin May 3, 2026
14996a7
chore(stt): add MODULATE_API_KEY to .env.template (#7140)
beastoin May 3, 2026
a7ccfb6
chore(helm): add MODULATE_API_KEY to backend-listen values (#7140)
beastoin May 3, 2026
720607c
chore(helm): add MODULATE_API_KEY to pusher values (#7140)
beastoin May 3, 2026
a85cc6c
chore(helm): add MODULATE_API_KEY to backend-secrets ExternalSecret (…
beastoin May 3, 2026
ae1e420
fix(stt): address review feedback on SafeModulateSocket (#7140)
beastoin May 3, 2026
7e06a5a
fix(stt): move Modulate EOS drain into receive_data finally block (#7…
beastoin May 3, 2026
20013a9
fix(helm): remove MODULATE_API_KEY from pusher charts (#7140)
beastoin May 3, 2026
ad83bff
fix(stt): yield before EOS to flush pending send() callbacks (#7140)
beastoin May 3, 2026
112dcb4
test(stt): add send-then-drain ordering regression test (#7140)
beastoin May 3, 2026
591a6f6
fix: SafeModulateSocket thread safety for send() header and queue
beastoin May 3, 2026
eb3e829
test: add QueueFull and header lock thread safety tests
beastoin May 3, 2026
f293481
fix: move Modulate API key from URL query to X-API-Key header
beastoin May 3, 2026
828fce7
fix: websockets 12.0 compat, non-blocking EOS drain, language fallthr…
beastoin May 3, 2026
11e6aa8
test: add language fallthrough test for dg->modulate routing
beastoin May 3, 2026
ffd432f
test: expand coverage for recv_loop, process_audio_modulate, prerecor…
beastoin May 3, 2026
255a2e4
test(stt): add visual evidence screenshots for PR #7142
beastoin May 3, 2026
e5bf3c2
fix: correct Modulate streaming protocol — s16le format, nested utter…
beastoin May 3, 2026
330b194
test: update tests for real Modulate API protocol (nested utterance, …
beastoin May 3, 2026
3296eef
Revert "test(stt): add visual evidence screenshots for PR #7142"
beastoin May 3, 2026
aa22b92
fix(stt): increase Modulate streaming drain wait from 5s to 10s
beastoin May 3, 2026
5dcf385
feat(stt): add pre-recorded benchmark script for Deepgram vs Modulate
beastoin May 3, 2026
59b6548
feat(stt): add streaming benchmark script for Deepgram vs Modulate
beastoin May 3, 2026
d704a68
feat(stt): add Suite 02 pre-recorded benchmark with real human speech
beastoin May 4, 2026
1e75df4
feat(stt): add Suite 02 streaming benchmark with real human speech
beastoin May 4, 2026
56c913f
feat(stt): enable Modulate real-time partial results for streaming
beastoin May 4, 2026
e0b46f5
fix(benchmark): strip punctuation from WER calculation, add punctuati…
beastoin May 4, 2026
e2f0e69
refactor(stt): make STT provider architecture provider-agnostic
beastoin May 4, 2026
518e5f9
refactor(stt): SafeDeepgramSocket inherits STTSocket, remove duck-typ…
beastoin May 4, 2026
0d1e332
refactor(stt): SafeModulateSocket inherits STTSocket, remove VAD from…
beastoin May 4, 2026
14d51ac
refactor(stt): GatedDeepgramSocket → GatedSTTSocket, DgWallMapper → W…
beastoin May 4, 2026
95c983d
refactor(stt): provider-agnostic VAD, dg_socket → stt_socket, univers…
beastoin May 4, 2026
a55dafe
test: update tests for STTSocket ABC and provider-agnostic renames
beastoin May 4, 2026
2d33c4f
fix(stt): normalize locale codes for Modulate language routing
beastoin May 4, 2026
8d009d7
fix: remove CRLF line endings from dev secrets values
beastoin May 4, 2026
0fccb6e
fix: remove CRLF line endings from prod secrets values
beastoin May 4, 2026
4aeeb44
test: add partial-result and locale routing tests for Modulate
beastoin May 4, 2026
4c3e7ae
test: add production-path coverage and file tuple assertions
beastoin May 4, 2026
fd40b20
test: properly await cancelled tasks to eliminate async warnings
beastoin May 4, 2026
80ea601
fix: remove undefined speech_profile_preseconds from Modulate STT path
beastoin May 5, 2026
17a94f8
test: add L2 listen API walkthrough script for STT provider comparison
beastoin May 5, 2026
5026157
test: save full transcripts in walkthrough JSON output
beastoin May 5, 2026
15a245c
fix(stt): reduce conversation_timeout in walkthrough to prevent cross…
beastoin May 5, 2026
7a6e653
fix(stt): rewrite Modulate partial handling for correct WER
beastoin May 5, 2026
272b01e
fix(stt): add pcm8 to pcm16 conversion for Omi device audio
beastoin May 5, 2026
e3e6970
fix(stt): correct walkthrough codec from pcm8 to pcm16
beastoin May 5, 2026
9eb6cfe
test(stt): update Modulate tests for new partial handling
beastoin May 5, 2026
d4262a0
fix(modulate-stt): stop sending empty EOS and flush partial on error
beastoin May 5, 2026
7b5cab2
fix(modulate-stt): disable VAD gate for Modulate provider
beastoin May 5, 2026
edab5f8
test(modulate-stt): update tests for EOS and error-flush behavior
beastoin May 5, 2026
41171e1
test(modulate-stt): add A/B comparison script for direct vs backend WER
beastoin May 5, 2026
521a062
fix(vad-gate): add passthrough_audio mode for STT providers with inte…
beastoin May 5, 2026
4a157ee
fix(modulate-stt): use VAD passthrough instead of disabling gate
beastoin May 5, 2026
8880d5d
test(modulate-stt): increase ready timeout for local dev Pusher retries
beastoin May 5, 2026
a432ad9
docs(modulate-stt): add minimal repro for non-deterministic utterance…
beastoin May 6, 2026
37d993c
chore(stt): remove hardcoded API key from A/B comparison script
beastoin May 6, 2026
483d72f
test(modulate-stt): add stability test proving non-deterministic WER
beastoin May 6, 2026
822d28a
test(modulate-stt): add silence compression test for VAD cost savings
beastoin May 6, 2026
834a334
test(modulate-stt): add silence sweep with baseline comparison
beastoin May 6, 2026
614fe4b
test(modulate-stt): add quick debug script for Modulate streaming
beastoin May 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions backend/.env.template
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ REDIS_DB_PORT=
REDIS_DB_PASSWORD=

DEEPGRAM_API_KEY=
MODULATE_API_KEY=

ADMIN_KEY=
OPENAI_API_KEY=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,11 @@ env:
secretKeyRef:
name: dev-omi-backend-secrets
key: DEEPGRAM_API_KEY
- name: MODULATE_API_KEY
valueFrom:
secretKeyRef:
name: dev-omi-backend-secrets
key: MODULATE_API_KEY
- name: FAL_KEY
valueFrom:
secretKeyRef:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,11 @@ env:
secretKeyRef:
name: prod-omi-backend-secrets
key: DEEPGRAM_API_KEY
- name: MODULATE_API_KEY
valueFrom:
secretKeyRef:
name: prod-omi-backend-secrets
key: MODULATE_API_KEY
- name: GOOGLE_MAPS_API_KEY
valueFrom:
secretKeyRef:
Expand Down
166 changes: 84 additions & 82 deletions backend/charts/backend-secrets/dev_omi_backend_secrets_values.yaml
Original file line number Diff line number Diff line change
@@ -1,82 +1,84 @@
gke:
projectID: based-hardware-dev
clusterLocation: us-central1
clusterName: dev-omi-gke

gsa:
name: dev-omi-backend-eso-gsa@based-hardware-dev.iam.gserviceaccount.com

serviceAccount:
name: dev-omi-backend-eso-ksa

externalSecret:
name: dev-omi-backend-external-secret
targetSecretName: dev-omi-backend-secrets
refreshInterval: 1h
secretKeys: # secretKey is the key in the Kubernetes secret, remoteKey is the key in the Secrets Manager
- secretKey: HUGGINGFACE_TOKEN
remoteKey: HUGGINGFACE_TOKEN
- secretKey: DEEPGRAM_API_KEY
remoteKey: DEEPGRAM_API_KEY
- secretKey: FAL_KEY
remoteKey: FAL_KEY
- secretKey: OPENAI_API_KEY
remoteKey: OPENAI_API_KEY
- secretKey: GOOGLE_MAPS_API_KEY
remoteKey: GOOGLE_MAPS_API_KEY
- secretKey: GITHUB_TOKEN
remoteKey: GITHUB_TOKEN
- secretKey: PINECONE_API_KEY
remoteKey: PINECONE_API_KEY
- secretKey: REDIS_DB_HOST
remoteKey: REDIS_DB_HOST
- secretKey: REDIS_DB_PASSWORD
remoteKey: REDIS_DB_PASSWORD
- secretKey: ADMIN_KEY
remoteKey: ADMIN_KEY
- secretKey: GOOGLE_APPLICATION_CREDENTIALS
remoteKey: GOOGLE_APPLICATION_CREDENTIALS
- secretKey: DD_API_KEY
remoteKey: DD_API_KEY
- secretKey: LANGCHAIN_API_KEY
remoteKey: LANGCHAIN_API_KEY
- secretKey: STRIPE_API_KEY
remoteKey: STRIPE_API_KEY
- secretKey: STRIPE_WEBHOOK_SECRET
remoteKey: STRIPE_WEBHOOK_SECRET
- secretKey: MARKETPLACE_APP_REVIEWERS
remoteKey: MARKETPLACE_APP_REVIEWERS
- secretKey: TYPESENSE_HOST
remoteKey: TYPESENSE_HOST
- secretKey: TYPESENSE_API_KEY
remoteKey: TYPESENSE_API_KEY
- secretKey: STT_SERVICE_MODELS
remoteKey: STT_SERVICE_MODELS
- secretKey: ENCRYPTION_SECRET
remoteKey: ENCRYPTION_SECRET
- secretKey: SERVICE_ACCOUNT_JSON
remoteKey: SERVICE_ACCOUNT_JSON
- secretKey: GEMINI_API_KEY
remoteKey: GEMINI_API_KEY
- secretKey: TWILIO_ACCOUNT_SID
remoteKey: TWILIO_ACCOUNT_SID
- secretKey: TWILIO_AUTH_TOKEN
remoteKey: TWILIO_AUTH_TOKEN
- secretKey: TWILIO_API_KEY_SID
remoteKey: TWILIO_API_KEY_SID
- secretKey: TWILIO_API_KEY_SECRET
remoteKey: TWILIO_API_KEY_SECRET
- secretKey: TWILIO_TWIML_APP_SID
remoteKey: TWILIO_TWIML_APP_SID
- secretKey: METRICS_SECRET
remoteKey: METRICS_SECRET
- secretKey: GROQ_API_KEY
remoteKey: GROQ_API_KEY
- secretKey: OPENROUTER_API_KEY
remoteKey: OPENROUTER_API_KEY
- secretKey: RAPID_API_HOST
remoteKey: RAPID_API_HOST
- secretKey: RAPID_API_KEY
remoteKey: RAPID_API_KEY
- secretKey: CONVERSATION_SUMMARIZED_APP_IDS
remoteKey: CONVERSATION_SUMMARIZED_APP_IDS
gke:
projectID: based-hardware-dev
clusterLocation: us-central1
clusterName: dev-omi-gke

gsa:
name: dev-omi-backend-eso-gsa@based-hardware-dev.iam.gserviceaccount.com

serviceAccount:
name: dev-omi-backend-eso-ksa

externalSecret:
name: dev-omi-backend-external-secret
targetSecretName: dev-omi-backend-secrets
refreshInterval: 1h
secretKeys: # secretKey is the key in the Kubernetes secret, remoteKey is the key in the Secrets Manager
- secretKey: HUGGINGFACE_TOKEN
remoteKey: HUGGINGFACE_TOKEN
- secretKey: DEEPGRAM_API_KEY
remoteKey: DEEPGRAM_API_KEY
- secretKey: MODULATE_API_KEY
remoteKey: MODULATE_API_KEY
- secretKey: FAL_KEY
remoteKey: FAL_KEY
- secretKey: OPENAI_API_KEY
remoteKey: OPENAI_API_KEY
- secretKey: GOOGLE_MAPS_API_KEY
remoteKey: GOOGLE_MAPS_API_KEY
- secretKey: GITHUB_TOKEN
remoteKey: GITHUB_TOKEN
- secretKey: PINECONE_API_KEY
remoteKey: PINECONE_API_KEY
- secretKey: REDIS_DB_HOST
remoteKey: REDIS_DB_HOST
- secretKey: REDIS_DB_PASSWORD
remoteKey: REDIS_DB_PASSWORD
- secretKey: ADMIN_KEY
remoteKey: ADMIN_KEY
- secretKey: GOOGLE_APPLICATION_CREDENTIALS
remoteKey: GOOGLE_APPLICATION_CREDENTIALS
- secretKey: DD_API_KEY
remoteKey: DD_API_KEY
- secretKey: LANGCHAIN_API_KEY
remoteKey: LANGCHAIN_API_KEY
- secretKey: STRIPE_API_KEY
remoteKey: STRIPE_API_KEY
- secretKey: STRIPE_WEBHOOK_SECRET
remoteKey: STRIPE_WEBHOOK_SECRET
- secretKey: MARKETPLACE_APP_REVIEWERS
remoteKey: MARKETPLACE_APP_REVIEWERS
- secretKey: TYPESENSE_HOST
remoteKey: TYPESENSE_HOST
- secretKey: TYPESENSE_API_KEY
remoteKey: TYPESENSE_API_KEY
- secretKey: STT_SERVICE_MODELS
remoteKey: STT_SERVICE_MODELS
- secretKey: ENCRYPTION_SECRET
remoteKey: ENCRYPTION_SECRET
- secretKey: SERVICE_ACCOUNT_JSON
remoteKey: SERVICE_ACCOUNT_JSON
- secretKey: GEMINI_API_KEY
remoteKey: GEMINI_API_KEY
- secretKey: TWILIO_ACCOUNT_SID
remoteKey: TWILIO_ACCOUNT_SID
- secretKey: TWILIO_AUTH_TOKEN
remoteKey: TWILIO_AUTH_TOKEN
- secretKey: TWILIO_API_KEY_SID
remoteKey: TWILIO_API_KEY_SID
- secretKey: TWILIO_API_KEY_SECRET
remoteKey: TWILIO_API_KEY_SECRET
- secretKey: TWILIO_TWIML_APP_SID
remoteKey: TWILIO_TWIML_APP_SID
- secretKey: METRICS_SECRET
remoteKey: METRICS_SECRET
- secretKey: GROQ_API_KEY
remoteKey: GROQ_API_KEY
- secretKey: OPENROUTER_API_KEY
remoteKey: OPENROUTER_API_KEY
- secretKey: RAPID_API_HOST
remoteKey: RAPID_API_HOST
- secretKey: RAPID_API_KEY
remoteKey: RAPID_API_KEY
- secretKey: CONVERSATION_SUMMARIZED_APP_IDS
remoteKey: CONVERSATION_SUMMARIZED_APP_IDS
162 changes: 82 additions & 80 deletions backend/charts/backend-secrets/prod_omi_backend_secrets_values.yaml
Original file line number Diff line number Diff line change
@@ -1,80 +1,82 @@
gke:
projectID: based-hardware
clusterLocation: us-central1
clusterName: prod-omi-gke

gsa:
name: prod-omi-backend-eso-gsa@based-hardware.iam.gserviceaccount.com

serviceAccount:
name: prod-omi-backend-eso-ksa

externalSecret:
name: prod-omi-backend-external-secret
targetSecretName: prod-omi-backend-secrets
refreshInterval: 1h
secretKeys: # secretKey is the key in the Kubernetes secret, remoteKey is the key in the Secrets Manager
- secretKey: HUGGINGFACE_TOKEN
remoteKey: HUGGINGFACE_TOKEN
- secretKey: GITHUB_TOKEN
remoteKey: GITHUB_TOKEN
- secretKey: OPENAI_API_KEY
remoteKey: OPENAI_API_KEY
- secretKey: FAL_KEY
remoteKey: FAL_KEY
- secretKey: GROQ_API_KEY
remoteKey: GROQ_API_KEY
- secretKey: PINECONE_API_KEY
remoteKey: PINECONE_API_KEY
- secretKey: REDIS_DB_HOST
remoteKey: REDIS_DB_HOST
- secretKey: REDIS_DB_PASSWORD
remoteKey: REDIS_DB_PASSWORD
- secretKey: ADMIN_KEY
remoteKey: ADMIN_KEY
- secretKey: DEEPGRAM_API_KEY
remoteKey: DEEPGRAM_API_KEY
- secretKey: GOOGLE_MAPS_API_KEY
remoteKey: GOOGLE_MAPS_API_KEY
- secretKey: GOOGLE_APPLICATION_CREDENTIALS
remoteKey: GOOGLE_APPLICATION_CREDENTIALS
- secretKey: LANGCHAIN_API_KEY
remoteKey: LANGCHAIN_API_KEY
- secretKey: DD_API_KEY
remoteKey: DD_API_KEY
- secretKey: STRIPE_API_KEY
remoteKey: STRIPE_API_KEY
- secretKey: STRIPE_WEBHOOK_SECRET
remoteKey: STRIPE_WEBHOOK_SECRET
- secretKey: MARKETPLACE_APP_REVIEWERS
remoteKey: MARKETPLACE_APP_REVIEWERS
- secretKey: TYPESENSE_HOST
remoteKey: TYPESENSE_HOST
- secretKey: TYPESENSE_API_KEY
remoteKey: TYPESENSE_API_KEY
- secretKey: RAPID_API_HOST
remoteKey: RAPID_API_HOST
- secretKey: RAPID_API_KEY
remoteKey: RAPID_API_KEY
- secretKey: OPENROUTER_API_KEY
remoteKey: OPENROUTER_API_KEY
- secretKey: CONVERSATION_SUMMARIZED_APP_IDS
remoteKey: CONVERSATION_SUMMARIZED_APP_IDS
- secretKey: STT_SERVICE_MODELS
remoteKey: STT_SERVICE_MODELS
- secretKey: ENCRYPTION_SECRET
remoteKey: ENCRYPTION_SECRET
- secretKey: GEMINI_API_KEY
remoteKey: GEMINI_API_KEY
- secretKey: TWILIO_ACCOUNT_SID
remoteKey: TWILIO_ACCOUNT_SID
- secretKey: TWILIO_AUTH_TOKEN
remoteKey: TWILIO_AUTH_TOKEN
- secretKey: TWILIO_API_KEY_SID
remoteKey: TWILIO_API_KEY_SID
- secretKey: TWILIO_API_KEY_SECRET
remoteKey: TWILIO_API_KEY_SECRET
- secretKey: TWILIO_TWIML_APP_SID
remoteKey: TWILIO_TWIML_APP_SID
- secretKey: METRICS_SECRET
remoteKey: METRICS_SECRET
gke:
projectID: based-hardware
clusterLocation: us-central1
clusterName: prod-omi-gke

gsa:
name: prod-omi-backend-eso-gsa@based-hardware.iam.gserviceaccount.com

serviceAccount:
name: prod-omi-backend-eso-ksa

externalSecret:
name: prod-omi-backend-external-secret
targetSecretName: prod-omi-backend-secrets
refreshInterval: 1h
secretKeys: # secretKey is the key in the Kubernetes secret, remoteKey is the key in the Secrets Manager
- secretKey: HUGGINGFACE_TOKEN
remoteKey: HUGGINGFACE_TOKEN
- secretKey: GITHUB_TOKEN
remoteKey: GITHUB_TOKEN
- secretKey: OPENAI_API_KEY
remoteKey: OPENAI_API_KEY
- secretKey: FAL_KEY
remoteKey: FAL_KEY
- secretKey: GROQ_API_KEY
remoteKey: GROQ_API_KEY
- secretKey: PINECONE_API_KEY
remoteKey: PINECONE_API_KEY
- secretKey: REDIS_DB_HOST
remoteKey: REDIS_DB_HOST
- secretKey: REDIS_DB_PASSWORD
remoteKey: REDIS_DB_PASSWORD
- secretKey: ADMIN_KEY
remoteKey: ADMIN_KEY
- secretKey: DEEPGRAM_API_KEY
remoteKey: DEEPGRAM_API_KEY
- secretKey: MODULATE_API_KEY
remoteKey: MODULATE_API_KEY
- secretKey: GOOGLE_MAPS_API_KEY
remoteKey: GOOGLE_MAPS_API_KEY
- secretKey: GOOGLE_APPLICATION_CREDENTIALS
remoteKey: GOOGLE_APPLICATION_CREDENTIALS
- secretKey: LANGCHAIN_API_KEY
remoteKey: LANGCHAIN_API_KEY
- secretKey: DD_API_KEY
remoteKey: DD_API_KEY
- secretKey: STRIPE_API_KEY
remoteKey: STRIPE_API_KEY
- secretKey: STRIPE_WEBHOOK_SECRET
remoteKey: STRIPE_WEBHOOK_SECRET
- secretKey: MARKETPLACE_APP_REVIEWERS
remoteKey: MARKETPLACE_APP_REVIEWERS
- secretKey: TYPESENSE_HOST
remoteKey: TYPESENSE_HOST
- secretKey: TYPESENSE_API_KEY
remoteKey: TYPESENSE_API_KEY
- secretKey: RAPID_API_HOST
remoteKey: RAPID_API_HOST
- secretKey: RAPID_API_KEY
remoteKey: RAPID_API_KEY
- secretKey: OPENROUTER_API_KEY
remoteKey: OPENROUTER_API_KEY
- secretKey: CONVERSATION_SUMMARIZED_APP_IDS
remoteKey: CONVERSATION_SUMMARIZED_APP_IDS
- secretKey: STT_SERVICE_MODELS
remoteKey: STT_SERVICE_MODELS
- secretKey: ENCRYPTION_SECRET
remoteKey: ENCRYPTION_SECRET
- secretKey: GEMINI_API_KEY
remoteKey: GEMINI_API_KEY
- secretKey: TWILIO_ACCOUNT_SID
remoteKey: TWILIO_ACCOUNT_SID
- secretKey: TWILIO_AUTH_TOKEN
remoteKey: TWILIO_AUTH_TOKEN
- secretKey: TWILIO_API_KEY_SID
remoteKey: TWILIO_API_KEY_SID
- secretKey: TWILIO_API_KEY_SECRET
remoteKey: TWILIO_API_KEY_SECRET
- secretKey: TWILIO_TWIML_APP_SID
remoteKey: TWILIO_TWIML_APP_SID
- secretKey: METRICS_SECRET
remoteKey: METRICS_SECRET
Loading
Loading