Repository navigation
ci: reduce the PR run time - #1063
Open
mbrandenburger wants to merge 10 commits into
Open
mbrandenburger wants to merge 10 commits into
mbrandenburger wants to merge 10 commits into
Conversation
The test jobs restored the build job's Go cache, which only holds what make build compiles. The suites compile other packages with other flags (-race, fpc_cgo, coverage), so each run downloaded the integration modules and rebuilt them, e.g. fsccli with -race for 103s in go_echo. Each test job now has its own cache, keyed on the suite and go.sum, so it caches exactly what it compiles. Only push runs save it (upstream that is main); PR runs restore main's. The repo's caches are already at the 10 GB quota, so PR branches saving 13 copies each would evict them. Refs hyperledger#1043 Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
go_simulation and test_network run their chaincode without SGX, so they need only Go and Docker. go_simulation already ran natively as well; it now runs only there, and test_network moves to the native matrix. The container unit job runs only cgo-unit-test (the C/C++ code, and internal and ercc with fpc_cgo) and the C++ linter. The Go unit tests, Go linter and license check already ran in native (unit). make suites lists the suites CI runs on each side; make test still runs unit tests only natively, and everything with FPC_CGO=1. Refs hyperledger#1043 Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
Four FSC topologies set the same Fabric log spec (gRPC at error), echo's none. EnableFPC, which every FPC topology goes through, now sets it as fpctopo.LogSpec, unless the topology set its own spec (as the simple testing network sample does). The topologies drop their copies. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
The FSC suites and irb pulled fabric-ccenv and fabric-baseos 2.5 and tagged them latest. FSC needs them only for chaincode the peers build themselves, which these topologies don't have: FPC chaincode runs as its own container. irb keeps pulling redis, which its experimenter test uses. The simple testing network sample, not run in CI, moves to ccenv 3.1. Each container job pulled the Fabric 3.1.4 images; only deployment uses them (its peer builds a plain Go chaincode in fabric-ccenv), the others run the Fabric binaries from the build outputs. irb's go test now gets the FPC log spec in FABRIC_LOGGING_SPEC, like the other FSC suites, so the peer CLI calls FSC makes log gRPC at error. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
The dev image cloned Fabric's whole history; make -C fabric patch only needs the release tag and its tree. A shallow clone takes about a third of the time and 155 MB less. The apt step is skipped when APT_ADD_PKGS is empty, as in CI. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
Every test job downloaded and loaded the 920 MB of images the build job saved (the dev image, ccenv and base-dev). Unless utils/docker changed, the build job pulls ccenv and base-dev from ghcr.io anyway; the test jobs now do the same and build the thin dev image on top (about 50s, against the image download plus about 40s of docker load). When utils/docker changed, the build job rebuilds the images and ships them as before, now as their own artifact, which publish also uses. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
Every invocation the C++ suites wait for (--waitForEvent, and the client SDK's submits) waits for the orderer to cut a block: up to the 2s batch timeout of the test network config. 1s is what the FSC test networks use; a block still holds every transaction that arrives within it. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
peer.sh installed ercc with a sleep 3 after each lifecycle step, 18s per ledger setup, which every C++ suite runs once. None waits for anything the CLI or the peer doesn't already: install and the queries are synchronous, approveformyorg and commit wait for their commit (--waitForEvent defaults to true in Fabric 3.x), and the peer holds the first call to ercc until it has launched. querycommitted at the end still checks that ercc is live. The sleep after channel join in ledger_init stays: there the peer has to catch up on the new channel first. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
The C++ suites never set FABRIC_LOGGING_SPEC, so every Fabric process they start (peer, orderer, each peer CLI call, the client SDK test) logged each gRPC connection step at info: about 23,000 lines per run, 16,000 of them in client_sdk. common_utils.sh, which every suite and peer.sh source, now defaults it to gRPC at error; a caller's own spec is kept. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
mbrandenburger
force-pushed
the
ci/phase3-polish
branch
from
November 6, 2026 01:35
a2a96e3 to
816ac22
Compare
…code deploys The FPC NWO extension slept 10s after committing each FPC chaincode, 3s after starting each chaincode container, and 3s at the end of PostRun: about 25 to 40s per FSC test network, in every FSC suite. As in FSC's PostRun (hyperledger-labs/fabric-smart-client#2321), it now waits for what the sleeps stood in for: - CommitChaincode already returns only once every peer has the definition committed, so the 10s sleep after it is gone. - After starting a chaincode container, it polls until the chaincode server accepts connections on its port, bounded by EventuallyTimeout. The peer dials it only on the first invocation. - The final sleep is gone: initEnclave has already invoked the enclave by then. Signed-off-by: Marcus Brandenburger <bur@zurich.ibm.com>
mbrandenburger
marked this pull request as ready for review
November 6, 2026 02:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
The median
FPC testrun over the last week was 12.8 min (32 successful runs). Its critical path is the build job (about 4 min), then the slowest container suite (go_echoorirb, about 7.5 min). In those suites the time went to Go compiles the cache should have saved, to downloading and loading a 1.3 GB artifact (920 MB of it images), to pulling Fabric images the suite doesn't use, and to fixed waits. Ten commits, best reviewed one by one:make buildcompiles; the suites compile other packages with other flags (-race,fpc_cgo, coverage) and started cold, e.g. 103s buildingfsccliwith-raceingo_echo. Each test job now has its own cache, keyedgo-<suite>-<hash of all go.sum>. Only push runs save it (upstream that's main) and PR runs restore main's, so PR branches don't fill the repo's 10 GB cache quota, which it is already near.go_simulationandtest_networkrun their chaincode without SGX.go_simulationalready ran natively too; it now runs only there, andtest_networkmoves to the native matrix (ubuntu-latesthas theyq4 andjqits scripts use). The containerunitjob runs onlycgo-unit-test(the C/C++ code, plusinternalanderccwithfpc_cgo) and the C++ linter; the Go unit tests, Go linter and license check already ran innative (unit).make testkeeps its meaning: unit tests only natively, everything withFPC_CGO=1.EnableFPC. Four FSC topologies set the same Fabric log spec (gRPC at error), echo's none.EnableFPCnow sets it asfpctopo.LogSpec, unless a topology set its own (the simple testing network sample does). irb'sgo testalso exports it inFABRIC_LOGGING_SPEC, like the other FSC suites, as FSC doesn't pass the topology spec to thediscoverCLI calls it makes.fabric-ccenv/fabric-baseos:2.5. FSC needs them only for chaincode the peers build themselves, which FPC topologies don't have, as FPC chaincode runs as its own container; the pulls are gone. The simple testing network sample (not in CI) moves toccenv:3.1. Of the container jobs, onlydeploymentstill pulls the Fabric 3.1.4 images: its peer builds a plain Go chaincode infabric-ccenv, the other suites run the Fabric binaries from the build outputs.make -C fabric patchneeds only the release tag. The clone takes about a third of the time (5.7s against 16.9s) and 155 MB less; the dev image builds in about 9s instead of 17s. The apt step is skipped whenAPT_ADD_PKGSis empty.utils/docker/**changed, the build job pullsccenvandbase-devfrom ghcr.io; the test jobs now do the same and build the dev image on top, instead of downloading anddocker loading 920 MB. Whenutils/docker/**changed, the images are rebuilt and shipped as before, as their own artifact, whichpublishalso uses.sgx/integration/config/configtx.yaml; 1s is what the FSC test networks use.peer.sh's ercc install. Sixsleep 3, 18s per ledger setup in every C++ suite. None waits for anything the CLI or the peer doesn't already: install and the queries are synchronous,approveformyorgandcommitwait for their commit (--waitForEventdefaults to true in Fabric 3.x), and the peer holds the first call to ercc until it has launched. Thesleep 3afterchannel joininledger_initstays.FABRIC_LOGGING_SPEC, so every Fabric process they start (peer, orderer, each peer CLI call, the client SDK test) logged each gRPC connection step at info: about 23,000 lines per run, 16,000 of them inclient_sdk.fabric/bin/lib/common_utils.sh, which every suite andpeer.shsource, now defaults it to gRPC at error; a caller's own spec is kept. Not covered: about 190 lines intest_networkfrom Fabric's own test-network scripts, and 49 from a gRPC server in one client SDK unit test.extension/fsc/integration/nwo/fabric/fpc/fpc.go) slept 10s after committing each FPC chaincode, 3s after starting each chaincode container, and 3s at the end ofPostRun: about 25 to 40s per FSC test network, in all 9 FSC jobs. As in FSC'sPostRun(test(nwo): wait for conditions instead of fixed sleeps when a Fabric network starts hyperledger-labs/fabric-smart-client#2321, already in the FSC v0.27.0 FPC uses), it now waits for the conditions instead.CommitChaincodealready returns once every peer has the definition committed, so the 10s sleep goes. After starting a container, it polls a TCP dial to the chaincode's port, bounded byEventuallyTimeout. The final sleep goes, asinitEnclavehas already invoked the enclave by then.Which issue(s) this PR fixes:
Refs #1043. Part of #1044.
Special notes for your reviewer:
utils/docker/**, so this PR's own runs rebuild the base images and take the "ship the images" path of commit 6. The pull path and main's Go caches only take effect after the merge, oncepublishhas pushed the new images and a push to main has saved the caches. I'll measure both on my fork (a run that fills the caches, then one that uses them) and add the numbers here.go_echo8.0 to 4.2 min,irb6.6 to 4.5,unit5.3 to 2.8,go_kv5.4 to 2.9. The C++-only suites don't change. The 13 caches take about 3.2 GB in total (go_echo608 MB,irb533 MB,unit459 MB), against 187 MB for the build job's cache before; when the quota is full, GitHub evicts the oldest first, mostly stalesetup-gocaches of PR branches.echo67s to 38s,client_sdk173s to 130s. If a suite ever misses an image (4), the pull comes back with3.1; if the ercc install races (8), the sleeps become polling onquerycommittedrather than coming back. The same goes for commit 10: if an FSC suite fails right after a chaincode commit,QueryInstalledReferencesgets polled rather than the 10s sleep coming back.:main, not the build job's digest, so apublishon main between the two pulls would hand a test job the newerbase-dev.client_sdkstays in the dev container: it deploys the C++ samples (fpc-c), so it needs the C++ enclaves and the SGX SDK runtime.