feat(snapshot): add concurrency option for parallel processing of snapshots by rushabhcodes · Pull Request #2105 · tscircuit/cli

rushabhcodes · 2026-02-24T13:07:47Z

This pull request adds support for concurrent snapshot generation in the CLI and improves error handling and reporting for snapshot operations. It introduces a new --concurrency option to the snapshot command, allowing users to specify how many files to process in parallel. The snapshot logic has been refactored to handle concurrency and to aggregate errors more robustly. Additionally, new tests have been added to verify concurrency behavior and error handling.

CLI and API enhancements:

Added a --concurrency option to the tsci snapshot CLI command, allowing users to specify the number of files to build in parallel (default: 1). The option is parsed and passed as a parameter to the snapshot logic. [1] [2]
Updated the SnapshotOptions type and the snapshotProject function to accept and handle a concurrency parameter, defaulting to sequential processing if not set. [1] [2]

Snapshot processing and error handling:

Refactored file processing in snapshotProject to support concurrent execution, process files in chunks based on the concurrency value, and aggregate errors and mismatches across all files. Errors are now collected per file and reported collectively at the end. [1] [2] [3] [4]
Improved error messages and handling throughout the snapshot process, ensuring that failures in individual files do not halt the entire process unless necessary, and that all errors are reported in a user-friendly way. [1] [2] [3]

Testing:

Added tests to verify that the --concurrency option processes multiple files in parallel, defaults to sequential when not set, and handles errors correctly when some files fail. [1] [2] [3]

…pshots

Copilot

Pull request overview

Adds a --concurrency flag to the tsci snapshot CLI command and refactors snapshot generation to process multiple circuit/board files concurrently while aggregating per-file errors/mismatches.

Changes:

Add --concurrency <number> option to the snapshot CLI command and pass it through to snapshot generation.
Refactor snapshotProject to support chunked parallel processing and collect errors/mismatches across all files.
Add (currently skipped) CLI tests intended to cover concurrency behavior and error handling.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`cli/snapshot/register.ts`	Adds CLI flag parsing for `--concurrency` and passes it to `snapshotProject`.
`lib/shared/snapshot-project.ts`	Implements concurrency via chunked `Promise.all`, per-file processing, and aggregated error reporting.
`tests/cli/snapshot/snapshot-concurrency01.test.ts`	New (skipped) test for multi-file processing with concurrency.
`tests/cli/snapshot/snapshot-concurrency02.test.ts`	New (skipped) test for default sequential behavior when concurrency isn’t provided.
`tests/cli/snapshot/snapshot-concurrency03.test.ts`	New (skipped) test for mixed success/failure behavior under concurrency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-24T13:11:16Z

cli/snapshot/register.ts

+        const concurrencyValue = Math.max(
+          1,
+          Number.parseInt(options.concurrency || "1", 10),
+        )


Math.max(1, Number.parseInt(...)) returns NaN if the user passes a non-numeric --concurrency value (e.g. --concurrency foo). Passing NaN through can break snapshot processing (e.g. infinite loop / chunking issues). Validate the parsed value with Number.isFinite(...) (and ideally Number.isInteger(...)) and fall back to 1 or throw a CLI validation error when invalid.

Suggested change

const concurrencyValue = Math.max(

1,

Number.parseInt(options.concurrency || "1", 10),

)

const parsedConcurrency = Number.parseInt(options.concurrency || "1", 10)

const concurrencyValue =

Number.isFinite(parsedConcurrency) &&

Number.isInteger(parsedConcurrency) &&

parsedConcurrency >= 1

? parsedConcurrency

: 1

Copilot · 2026-02-24T13:11:16Z

lib/shared/snapshot-project.ts

@@ -291,6 +351,13 @@ export const snapshotProject = async ({
    return onExit(0)
  }


In --update mode this function returns onExit(0) unconditionally, even if one or more files produced errors (which are collected/printed earlier). This makes tsci snapshot --update report success despite failures and contradicts the intended behavior in the new concurrency error-handling tests. Consider checking allErrors.length before the update early-return (or returning non-zero from the update branch when errors occurred), and avoid printing a success message when there were errors.

Copilot · 2026-02-24T13:11:17Z

lib/shared/snapshot-project.ts

+  if (concurrency <= 1) {
+    for (const file of boardFiles) {
+      const result = await processFile(file)
+      didUpdate = didUpdate || result.didUpdate
+      mismatches.push(...result.mismatches)
+      allErrors.push(...result.errors)
+
+      if (result.errors.length > 0) {
+        for (const err of result.errors) {
+          onError(kleur.red(`\n❌ ${err}\n`))
+        }
+      }
+    }
+  } else {
+    console.log(
+      `Processing ${boardFiles.length} file(s) with concurrency ${concurrency}...`,
+    )
+
+    const chunks: string[][] = []
+    for (let i = 0; i < boardFiles.length; i += concurrency) {
+      chunks.push(boardFiles.slice(i, i + concurrency))
+    }


concurrency is assumed to be a finite positive integer. If concurrency is NaN, Infinity, or a non-integer (possible if snapshotProject is called programmatically), the chunking loop for (let i = 0; i < boardFiles.length; i += concurrency) can behave incorrectly (including never terminating). Add a defensive normalization/validation step inside snapshotProject (e.g. coerce invalid values to 1 or throw) before using it in comparisons and loop increments.

Copilot · 2026-02-24T13:11:17Z

tests/cli/snapshot/snapshot-concurrency01.test.ts

+  </board>
+)`
+
+test.skip("snapshot with --concurrency builds multiple files in parallel", async () => {


This new test is currently test.skip(...), so the new concurrency behavior isn’t exercised in CI. If this is meant to validate the feature, it should be enabled (or gated behind an explicit env flag and documented) so regressions are caught.

Suggested change

test.skip("snapshot with --concurrency builds multiple files in parallel", async () => {

test("snapshot with --concurrency builds multiple files in parallel", async () => {

Copilot · 2026-02-24T13:11:17Z

tests/cli/snapshot/snapshot-concurrency02.test.ts

+test.skip("snapshot without --concurrency defaults to sequential", async () => {
+  const { tmpDir, runCommand } = await getCliTestFixture()
+
+  await Bun.write(join(tmpDir, "test.circuit.tsx"), circuitCode)


This new test is currently test.skip(...), so the default/sequential behavior for the new --concurrency option isn’t actually verified. Consider enabling it (or gating behind an explicit env flag) so it provides real coverage.

Copilot · 2026-02-24T13:11:17Z

tests/cli/snapshot/snapshot-concurrency03.test.ts

+test.skip("snapshot with --concurrency handles errors correctly", async () => {
+  const { tmpDir, runCommand } = await getCliTestFixture()
+
+  await Bun.write(join(tmpDir, "valid.circuit.tsx"), validCircuitCode)
+  await Bun.write(join(tmpDir, "invalid.circuit.tsx"), invalidCircuitCode)


This new test is currently test.skip(...), so the improved error aggregation/exit code behavior under concurrency isn’t verified. Since this feature changes CLI exit codes and error handling, enabling this test (or gating it behind an explicit env flag) would help prevent regressions.

Copilot · 2026-02-24T13:11:18Z

lib/shared/snapshot-project.ts

    const relativeFilePath = path.relative(projectDir, file)
+    const fileDidUpdate = false
+    const fileMismatches: string[] = []
+    const fileErrors: string[] = []


fileDidUpdate is initialized but never updated, and didUpdateForFile is the variable that actually tracks writes later. This makes the early-return objects a bit confusing. Consider removing fileDidUpdate and returning didUpdate: false on early errors, or define a single didUpdateForFile at the top and reuse it for all returns.

imrishabh18

You should not be using Promise.all, we already have the implementation of worker pools, you should use that.

Also, link to the issue or the discord message to give more context of why this PR. The AI descriptions are not at all helpful

rushabhcodes · 2026-02-24T16:44:18Z

I will use worker pools, this pr is an effort to decrease the the snapshot time in sparkfun-board repo, currently it takes 25 to 30 minutes to complete the snapshot test

seveibar

This is so much code, i dont think we want to make 1000 line change every time we want to do something concurrently ie its better to have a reusable pattern such as an rpc cli interface or something

- Removed the existing snapshot worker and worker pool implementations. - Introduced a new RPC worker architecture for handling snapshot operations. - Created `rpc-worker-entrypoint.ts` to manage RPC calls and service loading. - Implemented `build-service.ts` and `snapshot-service.ts` for build and snapshot functionalities. - Updated `snapshotProject` function to utilize the new RPC worker pool for processing snapshot jobs. - Adjusted the build script to point to the new RPC worker entry point.

github-actions · 2026-02-27T02:20:04Z

This PR has been automatically marked as stale because it has had no recent activity. It will be closed if no further activity occurs.

feat(snapshot): add concurrency option for parallel processing of sna…

1a12bd9

…pshots

Copilot AI review requested due to automatic review settings February 24, 2026 13:07

Copilot started reviewing on behalf of rushabhcodes February 24, 2026 13:08 View session

Copilot AI reviewed Feb 24, 2026

View reviewed changes

imrishabh18 requested changes Feb 24, 2026

View reviewed changes

feat(snapshot): implement worker pool for concurrent snapshot processing

37820d9

rushabhcodes requested a review from imrishabh18 February 24, 2026 16:54

seveibar reviewed Feb 24, 2026

View reviewed changes

github-actions bot added the no-pr-activity label Feb 27, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(snapshot): add concurrency option for parallel processing of snapshots#2105

feat(snapshot): add concurrency option for parallel processing of snapshots#2105
rushabhcodes wants to merge 3 commits intotscircuit:mainfrom
rushabhcodes:add-snapshot-concurrency

rushabhcodes commented Feb 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

imrishabh18 left a comment •

edited

Loading

Uh oh!

rushabhcodes commented Feb 24, 2026

Uh oh!

seveibar left a comment

Uh oh!

github-actions bot commented Feb 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

-        const concurrencyValue = Math.max(
-,
-          Number.parseInt(options.concurrency || "1", 10),
-        )
+        const parsedConcurrency = Number.parseInt(options.concurrency || "1", 10)
+        const concurrencyValue =
+          Number.isFinite(parsedConcurrency) &&
+          Number.isInteger(parsedConcurrency) &&
+          parsedConcurrency >= 1
+            ? parsedConcurrency
+            : 1

		@@ -291,6 +351,13 @@ export const snapshotProject = async ({
		return onExit(0)
		}

	test.skip("snapshot with --concurrency builds multiple files in parallel", async () => {
	test("snapshot with --concurrency builds multiple files in parallel", async () => {

Conversation

rushabhcodes commented Feb 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

imrishabh18 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rushabhcodes commented Feb 24, 2026

Uh oh!

seveibar left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

imrishabh18 left a comment •

edited

Loading