Skip to content

FileFlows is back with Convert to MP4 on the partner API, and its conversions card - #85

Open
jerodfritz wants to merge 5 commits into
stagingfrom
feature/fileflows-partner-api
Open

jerodfritz wants to merge 5 commits into
stagingfrom
feature/fileflows-partner-api

Conversation

@jerodfritz

@jerodfritz jerodfritz commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🔍 Problem

FileFlows 26.10 serves its REST API only to a licensed install with the API switched on and signs every request, so a free install answered every /api route with 404; the Convert to MP4 action and the API-reading card were removed (#72), and #75 brought the app back without them. Reven Software has since agreed to accept a credential HexOS issues: FileFlows 26.10.3+ reads PARTNER_API_KEY from its env, verifies it at startup with the HexOS public key, and serves its API to requests signed with it.

🛠️ Change

  • fileflows.json (script 1.1.0): the env entry PARTNER_API_KEY: "$PARTNER_TOKEN()", the convert-to-mp4 file action back, and the conversions card back as Tautulli, Plex to Tautulli, Emby now playing, Scrutiny temperatures and FileFlows back without its card #75 first wrote it.
  • fileflows/convert_to_mp4.ts: every FileFlows call goes through ctx.partnerRequest / ctx.partnerRequestJson; the script never holds the credential. Rewritten around one rule, the user's media is never replaced or deleted, with FileFlows' own queue as the one record of work in progress:
    • FileFlows is probed first; a 404 (a build before 26.10.3, or not restarted since the token was written) is explained to the user.
    • A file FileFlows is processing or holds queued, for any flow, refuses the run before anything is written.
    • The flow is written from the script's definition on every run (over the existing one of that name), never trusted by name; the definition is the same for every run.
    • FileFlows copies the result into a hidden .hexos-converting folder beside the media under the copy's final name; the hook gives the copy its final name beside the source with a hard link, which the kernel refuses if the name is taken, so a collision fails the run naming the file instead of overwriting. The only thing the hook ever removes is the staging name of a copy that already carries its final name (same inode, checked right before the unlink); a staging path something already sits at refuses the run naming it, and a copy refused its final name stays in the staging folder and the user is told where.
    • A run enforces its own 24-hour deadline, bounds every request to 30 s, and fails explicitly when FileFlows reports a file done but no output exists.
    • The staging folder must be a plain directory (a symlink or a file of that name fails the run naming it); the staging copy is named after the source's whole name plus .mp4, so film.mkv and film.avi never share one; list-all is read page by page until a page brings nothing new; a queue entry FileFlows holds back (on hold, disabled, out of schedule) counts as busy; a record is one of the selected files when it names the container path manually-add recorded, so a library's own record (relative to the library) and a like-named file elsewhere never refuse the run; a source whose name plus .mp4 passes 255 bytes is refused up front; a scan that reaches the page cap fails the run instead of calling a file it never read not busy.
    • No per-file reservations: an earlier draft held exclusive directories with owner markers, takeover and release by rename and ownership fences, and every race the review found lived in that protocol. Two runs started on one file inside the window between the busy check and the enqueue both proceed; the first to finish names the copy and the other reports it, and nothing is lost either way.
  • fileflows/widget_conversions.ts: as Tautulli, Plex to Tautulli, Emby now playing, Scrutiny temperatures and FileFlows back without its card #75 first wrote it, plus the waiting count shown whenever nothing is converting.
  • _lib/contract.ts: PARTNER_TOKEN among the macros. _lib/validate.ts: the macro only inside app_values, with a complete invocation and a lower-case id or nothing. _lib/hook_context.ts: partnerRequest and partnerRequestJson mirrored from the platform.

⚠️ Known limits

  • Waits on the TrueNAS catalog carrying FileFlows 26.10.3 (the chart pins the image by digest); until then the action explains the refusal.
  • Needs the platform branch feature/partner-tokens (the macro, ctx.partnerRequest); an older box is served this script with the macro stripped and offers the action without the credential.
  • The flow's stream handling is unchanged since Tautulli, Plex to Tautulli, Emby now playing, Scrutiny temperatures and FileFlows back without its card #75: video to H.264, remux to MP4, audio as is. A source whose audio MP4 cannot carry (WMA, for one) fails in FileFlows and the user is pointed at its log; adding an audio node waits on a lab box to prove it (backlog).
  • The conversions card reads FileFlows' SQLite database, as Tautulli, Plex to Tautulli, Emby now playing, Scrutiny temperatures and FileFlows back without its card #75 wrote it; an install moved to an external database engine shows setup. Reading the API through the partner token instead is a follow-up (backlog).
  • A library converting the same file at the same time is not refused: its record names the file relative to the library, and it writes where the library says, never where this flow does.
  • A staging copy a run abandoned at its deadline left behind refuses the next run on that file until the user moves it; nothing is removed to make room.
  • No live FileFlows encode was run against a box; the partner side was proven with the preview build in a local container.

✅ Verification

Check Result
bun _lib/validate.ts ✅ 55 scripts, 0 errors, 0 warnings
_lib typecheck, and a strict standalone typecheck of both FileFlows scripts ✅ passed
FileFlows 26.10.3 preview accepts a HexOS-signed token: /api/help 200, signed /api/status 200, wrong key 401 (local Docker) ✅ passed
Cold Codex review against a severity rubric (consequence, not reachability): the first pass on the staging design found one P0 (a symlinked staging folder), two P1 (one list-all page read; two sources sharing a staging name) and two P2 (held queue entries; library-relative record names), all fixed; then three more cold passes, each on the fixed tree: the second found four P1 (audio MP4 cannot carry; a like-named file under another mount refusing the run; a 252-byte name whose staging name is too long; the card on an external database) and one P2 (the page cap), the third one P0 (a user's file at the staging path removed) and one P1 (a library's relative record refusing the run), all fixed or parked as product scope; the fourth, at ac8e18d39, reported no P0, P1 or P2 ✅ no P0 to P2
Convert to MP4 on a lab box through the platform branch ⚠️ not run

📚 Knowledge

The platform's _knowledge/features/apps.md ("Partner tokens") and _knowledge/features/partner-integrations.md describe the mechanism; the script author's reference is on the docs site (hexos-docs docs/partner-tokens).

…versions card

The install script writes PARTNER_API_KEY from $PARTNER_TOKEN(), the action calls FileFlows through ctx.partnerRequest and never replaces or deletes a user's file, and the card reads FileFlows' own database. Waits on the TrueNAS catalog carrying FileFlows 26.10.3.
…be the record of work in progress, instead of per-file reservations

The reservation protocol (exclusive directories, owner markers, takeover and release by rename, abandoned markers, ownership fences before every step) was the source of every race a review found, and file-system mutual exclusion across processes with crash recovery is not a problem this hook needs to solve. FileFlows refuses a run while it is processing or holds queued any selected file; the flow copies the result into .hexos-converting beside the media under the copy's final name; the hook links it to its final name, which the kernel refuses when the name is taken, and removes only its own staging copy. Two runs started on one file inside the window between the busy check and the enqueue both proceed; the first names the copy and the other reports it, and nothing beside the user's media is replaced or deleted.
…very list-all page, and count held queue entries and library-relative names as busy

The staging folder is refused when it is a symlink or a file, so the copy and the hook's one removal never leave it. The staging copy is named after the source file with .mp4 appended, so film.mkv and film.avi never share one. list-all answers one page, so pages are read until one brings nothing new. A queue entry FileFlows holds back (on hold, disabled, out of schedule) is work in progress and refuses the run, and a library's own record, which names the file relative to the library, matches the selected path by its tail.
… and fail when the library has more pages than the action reads

A record whose name is absolute is one of the selected files only when it is that path, so a like-named file under another mount no longer refuses the run; a relative name, a library's own record, still matches by its tail. A source whose name plus .mp4 passes 255 bytes is refused up front instead of failing inside FileFlows. A list-all scan that reaches its page cap fails the run rather than reporting a file not busy that it never read.
…s final name, and match a record by the path manually-add gave it

A staging path something already sits at refuses the run naming it, whatever put it there, instead of being cleared; a copy refused its final name stays in the staging folder and the user is told where. A library's own record, which names the file relative to the library, no longer matches a selected file, so a like-named file elsewhere cannot refuse the run; a library converting the same file writes where the library says. The card's comment on API access was stale.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant