Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 27 additions & 9 deletions .github/workflows/release-apk.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ name: Build Release APK

on:
push:
branches:
- "codex/moonshine-vocabulary-beta"
tags:
- "v*"
workflow_dispatch:
Expand All @@ -11,7 +13,7 @@ permissions:

jobs:
build:
name: Build dev debug APK
name: Build APK
runs-on: ubuntu-latest

steps:
Expand All @@ -20,6 +22,18 @@ jobs:
with:
submodules: recursive

- name: Configure release signing
env:
RELEASE_KEYSTORE_BASE64: ${{ secrets.RELEASE_KEYSTORE_BASE64 }}
RELEASE_KEY_ALIAS: ${{ secrets.RELEASE_KEY_ALIAS }}
RELEASE_KEY_PASSWORD: ${{ secrets.RELEASE_KEY_PASSWORD }}
RELEASE_STORE_PASSWORD: ${{ secrets.RELEASE_STORE_PASSWORD }}
run: |
test -n "$RELEASE_KEYSTORE_BASE64"
printf '%s' "$RELEASE_KEYSTORE_BASE64" | base64 --decode > release-signing.jks
printf 'storeFile=release-signing.jks\nkeyAlias=%s\nkeyPassword=%s\nstorePassword=%s\n' \
"$RELEASE_KEY_ALIAS" "$RELEASE_KEY_PASSWORD" "$RELEASE_STORE_PASSWORD" > keystore.properties

- name: Set up JDK
uses: actions/setup-java@v4
with:
Expand All @@ -43,34 +57,38 @@ jobs:
run: cargo install cargo-ndk

- name: Build APK
run: ./gradlew :app:assembleStandaloneRelease
run: |
./gradlew :app:assembleStandaloneRelease
echo "APK_DIR=app/build/outputs/apk/standalone/release" >> "$GITHUB_ENV"
echo "APK_NAME=futo-voice-input-moonshine" >> "$GITHUB_ENV"

- name: Rename APK
run: |
mkdir -p release
VERSION="${GITHUB_REF_NAME:-manual}"
APK_PATH="$(find app/build/outputs/apk/standalone/release -maxdepth 1 -type f -name '*.apk' | head -n 1)"
VERSION="${VERSION//\//-}"
APK_PATH="$(find "$APK_DIR" -maxdepth 1 -type f -name '*.apk' | head -n 1)"
if [ -z "$APK_PATH" ]; then
echo "No APK found in app/build/outputs/apk/standalone/release"
echo "No APK found in $APK_DIR"
exit 1
fi
cp "$APK_PATH" "release/futo-voice-input-parakeet-${VERSION}.apk"
cp "$APK_PATH" "release/${APK_NAME}-${VERSION}.apk"

- name: Upload APK artifact
uses: actions/upload-artifact@v4
with:
name: futo-voice-input-parakeet-apk
name: futo-voice-input-moonshine-apk
path: release/*.apk

- name: Create GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v2
with:
name: FUTO Voice Input Parakeet ${{ github.ref_name }}
name: FUTO Voice Input Moonshine ${{ github.ref_name }}
body: |
Runtime-download Parakeet release of the FUTO Voice Input fork, now branded as FUTO Voice Input Parakeet.
Offline Moonshine streaming release of the FUTO Voice Input fork, with Parakeet and Whisper retained as selectable backends.

This APK no longer packages the Parakeet ONNX model files directly. The app downloads the Parakeet model on first use or from the Model Options screen, which keeps the APK much smaller while preserving offline transcription after the model is installed.
Models download on first use or from Model Options, keeping the APK smaller while preserving offline transcription after installation.

The Parakeet integration and repository changes in this fork were built with AI assistance using Codex (GPT-5).
files: release/*.apk
Expand Down
22 changes: 18 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This is a personal fork of FUTO Voice Input. Upstreams:
- Original GitLab: `https://gitlab.futo.org/keyboard/voiceinput`
- Public fork: `https://github.com/Today20092/futo_with_parakeet`

The goal is to preserve FUTO Voice Input's Android IME/activity UX while making NVIDIA Parakeet TDT 0.6B V3 via ONNX Runtime the default offline recognizer. Legacy FUTO Whisper/GGML remains selectable from Model Options.
The goal is to preserve FUTO Voice Input's Android IME/activity UX while making Moonshine v2 Small Streaming the default offline recognizer. NVIDIA Parakeet TDT 0.6B V3 and legacy FUTO Whisper/GGML remain selectable from Model Options.

## High-Level Architecture

Expand All @@ -28,7 +28,7 @@ Primary runtime flow:
2. Both wrap `RecognizerView`, which owns the Compose recognition UI and forwards lifecycle events.
3. `RecognizerView` delegates recording/model work to `AudioRecognizer`.
4. `AudioRecognizer` records 16 kHz mono PCM with `AudioRecord`, applies WebRTC VAD, buffers float samples, then selects a `SpeechBackend`.
5. Default backend is `ParakeetBackend`; optional backend is `WhisperGGMLBackend`.
5. Default backend is `MoonshineBackend`; optional backends are `ParakeetBackend` and `WhisperGGMLBackend`.
6. `ParakeetBackend` calls `ParakeetNative`, which loads `c++_shared`, `onnxruntime`, and `parakeet_voiceinput`.
7. Rust JNI functions in `parakeet-native/src/lib.rs` call `parakeet-native/src/engine.rs`, which uses `transcribe-rs` and ONNX Runtime.

Expand All @@ -40,8 +40,8 @@ Primary runtime flow:
- `app/src/main/java/org/futo/voiceinput/RecognizerView.kt`: recognition UI state machine, sounds, permission/model prompts, result dispatch hooks.
- `app/src/main/java/org/futo/voiceinput/RecognizeActivity.kt`: speech recognizer activity entry point.
- `app/src/main/java/org/futo/voiceinput/VoiceInputMethodService.kt`: IME entry point, commits final/partial text to the active input connection, switches back on cancel.
- `app/src/main/java/org/futo/voiceinput/settings/Settings.kt`: DataStore keys. `SPEECH_BACKEND` defaults to `parakeet`.
- `app/src/main/java/org/futo/voiceinput/settings/pages/Models.kt`: Model Options UI for Parakeet vs Whisper/GGML.
- `app/src/main/java/org/futo/voiceinput/settings/Settings.kt`: DataStore keys. `SPEECH_BACKEND` defaults to `moonshine`.
- `app/src/main/java/org/futo/voiceinput/settings/pages/Models.kt`: Model Options UI for Moonshine, Parakeet, and Whisper/GGML.
- `app/src/main/java/org/futo/voiceinput/downloader/DownloadActivity.kt`: shared model downloader. Supports explicit file URLs/hashes for Parakeet and legacy FUTO model names for Whisper.
- `app/src/main/java/org/futo/voiceinput/parakeet/ParakeetModel.kt`: Parakeet file list, Hugging Face URLs, download marker, hash verification, model download intent.
- `app/src/main/java/org/futo/voiceinput/parakeet/ParakeetEngineManager.kt`: shared/warm Parakeet backend, idle unload timeout.
Expand Down Expand Up @@ -116,3 +116,17 @@ Generated APK names are customized in `app/build.gradle`, but the GitHub workflo
- Do not remove legacy Whisper/GGML paths unless explicitly requested; this fork intentionally keeps them as selectable fallback.
- Be careful with generated/native outputs under `app/src/main/jniLibs/`; they are produced by Gradle/Rust build tasks.
- This repo may have local uncommitted changes. Check `git status --short` before editing and avoid reverting user changes.

## Agent skills

### Issue tracker

Issues are tracked in GitHub Issues; external PRs are not a triage surface. See `docs/agents/issue-tracker.md`.

### Triage labels

Use the canonical labels `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, and `wontfix`. See `docs/agents/triage-labels.md`.

### Domain docs

This is a single-context repository. See `docs/agents/domain.md`.
43 changes: 21 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,61 +1,60 @@
# FUTO Voice Input Parakeet
# FUTO Voice Input Moonshine

This is a personal fork of FUTO Voice Input, renamed as FUTO Voice Input Parakeet, that keeps the FUTO voice keyboard experience and uses NVIDIA Parakeet TDT 0.6B V3 through ONNX Runtime by default. The legacy FUTO Whisper/GGML backend can also be selected from Model Options.
This personal fork keeps the FUTO voice keyboard experience and uses Moonshine v2 Small Streaming as its default offline recognizer. NVIDIA Parakeet TDT 0.6B V3 and the legacy FUTO Whisper/GGML backend remain selectable from Model Options.

This fork's Parakeet integration and repository changes were built with AI assistance using Codex (GPT-5).

The goal is straightforward: keep the FUTO UI, recording flow, dark theme support, VAD silence stopping, microphone animation, and keyboard switch-back behavior, while using Parakeet as the main recognizer because it is fast and accurate for English speech.
The goal is straightforward: keep the FUTO UI and recording flow while adding responsive streaming transcription and personal vocabulary corrections.

## What Changed

- FUTO Voice Input Parakeet remains the base Android app and UI.
- Parakeet is the default speech backend.
- The legacy Whisper/GGML recognition path can be selected as an optional backend.
- A Kotlin backend layer calls a Rust JNI library for Parakeet.
- The Rust library loads Parakeet ONNX models through `transcribe-rs` and ONNX Runtime.
- The Model Options screen lets you choose Parakeet or Whisper/GGML.
- Moonshine v2 Small Streaming is the default backend and emits live partial transcripts.
- Parakeet and legacy Whisper/GGML remain selectable backends.
- Batch and streaming recognizers share backend-neutral Kotlin contracts.
- Personal vocabulary entries correct partial and final transcripts; use `heard => preferred` for explicit aliases.
- The stable app uses the distinct `org.futo.voiceinput.moonshine` package ID.
- Only the selected backend's model files are required before voice input starts.

## Screenshots

### Model Options

<img src="docs/screenshots/model-options.png" alt="Model Options screen with Parakeet selected" width="360">
<img src="docs/screenshots/model-options.png" alt="Model Options screen" width="360">

Parakeet is selected as the default backend, with Whisper/GGML still available as a fallback.
Moonshine is selected by default, with Parakeet and Whisper/GGML available as fallbacks.

## Active Model

The default backend is:

```text
Unified Parakeet TDT 0.6B V3
Moonshine v2 Small Streaming English
```

The app downloads the ONNX model assets from:
The app downloads the quantized model assets from:

```text
https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx
https://download.moonshine.ai/model/small-streaming-en/quantized/
```

By default, the Parakeet model files are not packaged into the APK. This keeps the APK much smaller than the first bundled-model release.
Model files are downloaded on first use rather than packaged into the APK.

After installing the APK, the model is downloaded by the app:

1. Open FUTO Voice Input Parakeet Settings.
1. Open FUTO Voice Input Moonshine Settings.
2. Tap **Model**.
3. Tap **Unified Parakeet TDT 0.6B V3** under **Parakeet Model**.
3. Select **Moonshine v2 Small Streaming**.
4. Confirm the download.

If you try to use voice input before downloading the model, the app shows a download prompt before it starts recording. Once the download finishes, the app loads Parakeet from local app storage and transcription runs offline.
If you try voice input before downloading the model, the app prompts for the download. Transcription runs offline after installation.

Downloaded model files are stored in app-private storage:

```text
filesDir/parakeet-tdt-0.6b-v3-int8/
filesDir/moonshine-small-streaming-en/
```

If you select **Whisper/GGML** in Model Options, the app uses the legacy FUTO English and multilingual GGML model settings and downloader. Those GGML files are also stored in app-private storage and run offline after download. Parakeet downloads from Hugging Face, while Whisper/GGML models download through the existing FUTO model downloader.
Parakeet and Whisper/GGML models are also stored in app-private storage and run offline after download.

## Building Locally

Expand Down Expand Up @@ -131,8 +130,8 @@ You can also run the workflow manually from the Actions tab. Manual runs upload

- First supported ABI is `arm64-v8a`.
- This is intended for sideloading and personal testing.
- The normal APK does not include the Parakeet ONNX model files. The model downloads at runtime from Hugging Face.
- Parakeet currently returns a final transcript after recording stops; live partial transcripts are not implemented.
- The normal APK does not include speech model files.
- Moonshine emits live partial transcripts; Parakeet currently returns only a final transcript.
- The app requires network access to download the selected backend's model the first time, then transcription runs offline.

## Attribution And License
Expand Down
12 changes: 8 additions & 4 deletions app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ android {

defaultConfig {
applicationId "org.futo.voiceinput"
minSdk 24
minSdk 26
targetSdk 35
versionCode 35
versionName "1.3.7-8-parakeet-branding"
versionCode 36
versionName "1.4.0"
buildConfigField "boolean", "BUNDLE_PARAKEET_MODEL", bundleParakeetModel.toString()

testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
Expand Down Expand Up @@ -122,6 +122,7 @@ android {
jniLibs {
useLegacyPackaging false
keepDebugSymbols += '**/*.so'
pickFirsts += '**/libonnxruntime.so'
}
}

Expand All @@ -133,6 +134,8 @@ android {

standalone {
dimension "version"
applicationIdSuffix ".moonshine"
versionNameSuffix "-moonshine"
}

fDroid {
Expand Down Expand Up @@ -229,7 +232,7 @@ android {

android.applicationVariants.all { variant ->
variant.outputs.all {
def appSlug = "futo-voice-input-parakeet"
def appSlug = "futo-voice-input-moonshine"
def flavor = variant.flavorName ?: "unflavored"
def version = variant.versionName.replaceAll(/[^A-Za-z0-9._-]/, "-")
def buildType = variant.buildType.name
Expand Down Expand Up @@ -271,6 +274,7 @@ dependencies {
implementation 'com.squareup.okhttp3:okhttp:4.11.0'

implementation 'com.microsoft.onnxruntime:onnxruntime-android:1.22.0'
implementation 'ai.moonshine:moonshine-voice:0.0.68'

implementation 'ch.acra:acra-http:5.11.1'
implementation 'ch.acra:acra-dialog:5.11.1'
Expand Down
4 changes: 2 additions & 2 deletions app/src/dev/res/values/strings.xml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<resources>
<string name="app_name">FUTO Voice Input Parakeet [Dev]</string>
<string name="voice_input_method">FUTO Voice Input Parakeet Method [Dev]</string>
<string name="app_name">FUTO Voice Input Moonshine Beta</string>
<string name="voice_input_method">FUTO Voice Input Moonshine Method [Beta]</string>
</resources>
Loading
Loading