Skip to content

feat: remove stopped ingest tasks from the queue - #712

Open
annieyii wants to merge 1 commit into
nashsu:mainfrom
annieyii:feat/ingest-queue-remove-task
Open

annieyii wants to merge 1 commit into
nashsu:mainfrom
annieyii:feat/ingest-queue-remove-task

Conversation

@annieyii

Copy link
Copy Markdown
Contributor

Closes #707

What

Lets the user permanently remove stopped ingest tasks from the queue, one at a
time from the task row or in bulk from the existing selection bar.

  • removeTasks(taskIds) in ingest-queue.ts drops failed / cancelled tasks
    for the active project in a single queue write. removeTask(taskId) is a thin
    wrapper over it.
  • Pending and processing tasks are skipped, not force-stopped. Cancel them first,
    then remove. Nothing touches activeRuns, cancelledInFlightTaskIds or the
    commit coordinator, so an in-flight run keeps owning its own teardown.
  • Tasks with a live entry in activeRuns are skipped too. cancelled only
    records that cancellation was requested: the run holds its worker slot until
    it observes the AbortSignal, and a commit past the interruptible point still
    finishes. Removing the entry then would free the sourcePath that enqueueBatch
    dedupes against, so re-importing the same source would mint a fresh task ID
    that the !activeRuns.has(task.id) guard in processNext cannot match. The
    two runs cannot interleave their writes, since autoIngest holds a per-source
    project lock, but the newcomer would take a worker slot and sit in the panel
    as "processing" while it blocks on that lock; with a small worker pool the
    queue can look stalled while nothing is able to progress.
  • hasActiveRun(taskId) lets the panel hide the remove affordance for that beat
    rather than offering a click the core would refuse, which also keeps the bulk
    confirmation count honest.
  • retainQueuedSelection() reconciles the selection against the queue after a
    removal instead of clearing it, so a row that was not removed stays visible
    and selected. It returns the original Set when nothing changed.
  • Activity Panel: a trash button on every removable stopped row (hidden while
    a run is still winding down), and a "Remove" action in the bulk bar next to
    Restart / Cancel. The bulk action confirms first because the removal cannot be
    undone.
  • Two new key paths, common.remove and activity.removeSelectedConfirm, added
    across all four bundles. The row button previously reused common.dismiss,
    which translates to "hide" (关闭, Nascondi, Скрыть) and misdescribes a
    permanent action.

Why

#557 established that stopped tasks are retained on purpose so they can be
restarted, and 0fd9257 implemented that by turning cancel from "drop the task"
into "keep it as cancelled". That closed the recoverability gap but left no way
out for tasks that will never succeed: a corrupt PDF, an unsupported format, or a
source file the user has since deleted. A successful task leaves the queue as
soon as it commits, and saveQueue filters any stray done entry, so nothing
else accumulates. failed and cancelled entries stay in the Activity Panel
indefinitely. #707 reports exactly this.

Two removal paths already exist and neither fits. discardTasksForSources is
reachable only through scheduled-import reconciliation, never from the UI.
clearCompletedTasks drops every non-pending/processing task in one call, has no
production caller, and carries neither a selection nor a live-run guard, so
exposing it as-is would give up the granularity #707 asks for.

Relationship to the other open queue-cleanup PRs

scope statuses
#690 clear the whole queue, aborts in-flight work all
#704 bulk clear, part of a broad multi-feature PR cancelled only
this single row + bulk over the existing selection failed + cancelled

This one is the narrowest of the three: it never stops running work, and it
reuses the selection bar that #557 already added rather than introducing another
top-level button.

Worth flagging: #704 also adds an export async function removeTasks(taskIds: readonly string[]): Promise<number> immediately after discardTasksForSources,
so the two cannot be merged mechanically even though the behaviour differs
(#704 accepts cancelled only and has no live-run guard). #690 touches the same
seven files and currently shows as conflicting. Happy to rebase, rename, or fold
this into whichever of the others you prefer to land first.

Tests

src/lib/ingest-queue.test.ts:

  • removes a cancelled task, refuses a processing one
  • removes a failed task and asserts the persisted snapshot holds exactly the
    survivor, not merely that the removed ID is absent
  • bulk removal over a mixed selection removes only the stopped task; unknown IDs
    and an empty selection are no-ops
  • a cancelled task whose run has not exited yet is refused
  • retainQueuedSelection drops removed IDs, keeps refused and untouched ones,
    and preserves Set identity when nothing changed

Mutation checks: dropping the activeRuns guard turns the active-cancelled test
red, persisting an empty snapshot turns the survivor test red, and clearing the
selection outright or removing the Set identity short-circuit each turn the
reconciliation test red.

npm run test:mocks → 132 files, 1881 tests passing. npm run typecheck and
npm run build clean. The i18n parity test covers both new key paths.

Closes nashsu#707.

A queue full of stopped sources currently has no way out. nashsu#557 established
that stopped tasks are retained on purpose so they can be restarted, and
0fd9257 implemented that by turning cancel from "drop the task" into "keep
it as cancelled". That closed the recoverability gap but left nothing for
tasks that will never succeed: a corrupt PDF, an unsupported format, or a
source the user has since deleted. A successful task leaves the queue as
soon as it commits, and saveQueue filters any stray done entry, so nothing
else accumulates. Failed and cancelled entries stay in the Activity Panel
indefinitely.

Two removal paths already exist and neither fits. discardTasksForSources is
reachable only from scheduled-import reconciliation, never from the UI.
clearCompletedTasks drops every non-pending/processing task in one call, has
no production caller, and carries neither a selection nor a live-run guard,
so exposing it as-is would give up the granularity nashsu#707 asks for.

- removeTasks(taskIds) drops failed/cancelled tasks for the active project
  in one queue write; removeTask(taskId) wraps it for the single-row case.
  Pending and processing tasks are skipped, not force-stopped.
- Skip tasks with a live entry in activeRuns. A "cancelled" status only
  records that cancellation was requested: the run holds its worker slot
  until it observes the AbortSignal, and a commit past the interruptible
  point still finishes. Removing the entry then would free the sourcePath
  that enqueueBatch dedupes against, so re-importing the same source would
  mint a fresh task ID that the !activeRuns.has(task.id) guard in
  processNext cannot match. The two runs cannot interleave their writes,
  since autoIngest holds a per-source project lock, but the newcomer would
  take a worker slot and block on that lock; because commitCoordinator
  chains reservations in start order, a small worker pool can then look
  stalled while nothing is able to progress.
- hasActiveRun(taskId) lets the panel hide the remove affordance for that
  beat instead of offering a click the core would refuse, which also keeps
  the bulk confirmation count honest.
- retainQueuedSelection() reconciles the selection against the queue after
  a removal rather than clearing it, so a row that was not removed stays
  visible and selected. It returns the original Set when nothing changed.
- Activity Panel: a trash button on every removable row, and a Remove
  action in the selection bar next to Restart and Cancel. The bulk action
  confirms first, because it cannot be undone.
- Add common.remove (en/zh/it/ru). The row button first reused
  common.dismiss, which translates to "hide" (关闭, Nascondi, Скрыть) and
  misdescribes a permanent action. The confirm string keeps its count in a
  separate label so no wording depends on grammatical number.

Tests cover single and bulk removal, refusal of a processing task, refusal
of a cancelled task whose run has not exited, persistence of the surviving
queue, and selection reconciliation. Mutation checks: dropping the
activeRuns guard turns the active-cancelled test red, persisting an empty
snapshot turns the survivor test red, clearing the selection outright and
removing the Set identity short-circuit each turn the reconciliation test
red.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

摄取队列没有批量删除

1 participant