Add SpeechTranscriber as STT option to Assist#4384
Draft
Conversation
…nguage selection, and fallback mechanism Co-authored-by: bgoncal <5808343+bgoncal@users.noreply.github.com>
…chTranscriber Co-authored-by: bgoncal <5808343+bgoncal@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add local audio transcription feature with opt-in mechanism
Add opt-in local audio transcription via SFSpeechRecognizer
Feb 23, 2026
Co-authored-by: bgoncal <5808343+bgoncal@users.noreply.github.com>
Co-authored-by: bgoncal <5808343+bgoncal@users.noreply.github.com>
Copilot
AI
changed the title
Add opt-in local audio transcription via SFSpeechRecognizer
Use SpeechTranscriber as SPM dependency instead of copied implementation
Feb 23, 2026
Member
|
@copilot Build is failing, code needs to also be restricted to iOS17+ which is what SPM supports |
…ailable, add to Xcode project Co-authored-by: bgoncal <5808343+bgoncal@users.noreply.github.com>
Contributor
Author
Fixed in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds opt-in local audio transcription using the SpeechTranscriber SPM package as an alternative to the assist websocket API for speech-to-text.
SPM Integration:
bgoncal/SpeechTranscriberv1.0.0 as an Xcode SPM package dependencySpeechTranscriberAdapterwraps the SPM class, conforming toSpeechTranscriberProtocolfor DI/testabilityAudioRecorderor buffer forwarding needed for on-device STTiOS 17+ Availability:
SpeechTranscriberProtocol,SpeechTranscriberDelegate) are in a separateSpeechTranscriberProtocol.swiftfile with no SPM import, available on all iOS versionsSpeechTranscriberAdapteris gated with@available(iOS 17, *)since the SPM package requires iOS 17+NoOpSpeechTranscriberprovides a fallback for pre-iOS 17 devicesAssistView+Build.swiftuses#available(iOS 17, *)to conditionally select the implementationSettings & Configuration:
AssistConfigurationgainssttLanguagefield (persisted in GRDB with auto-migration)AssistSettingsViewadds on-device STT toggle + language picker fromSFSpeechRecognizer.supportedLocales(), with unsupported-language warningAssistViewModel Flow:
assistWithAudio()branches: on-device STT bypassesAudioRecorderentirely, server STT unchanged.text()source to the assist pipelineTests: 10 test cases covering enable/disable, language routing, text completion, empty input, failure fallback, and stop streaming.
Screenshots
Link to pull request in Documentation repository
Documentation: home-assistant/companion.home-assistant#
Any other notes
The adapter uses
nonisolatedprotocol conformance withTask { @MainActor in }bridging to satisfy the SPM package's@MainActorrequirement. Active tasks are tracked and cancelled to prevent race conditions on rapid start/stop.✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.