Skip to content

fix(ci): mirror runnable OCI images without unsupported referrers - #1012

Merged
XiaoSeS merged 1 commit into
mainfrom
fix/runtime-mirror-attestations-20261025
Oct 25, 2026
Merged

XiaoSeS merged 1 commit into
mainfrom
fix/runtime-mirror-attestations-20261025

Conversation

@XiaoSeS

@XiaoSeS XiaoSeS commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • 修复应用发布的运行依赖镜像同步失败:RustFS 上游 OCI index 包含证明描述符,阿里云目标仓库拒绝其 application/vnd.oci.empty.v1+json,返回403。
  • Buildx 会主动重新发现/复制 referrers,单独过滤源 index 或使用子manifest不能解决该失败。仅在依赖同步 job 中使用固定官方 regctl v0.11.7 并校验 SHA256,显式关闭 referrers 复制;四应用镜像的 Buildx 构建不变。
  • 保留所有已分类的可运行平台、变体及原始子manifest/config/layer内容;不限定为amd64。未知描述符、空index、无效JSON及复制失败均失败退出。
  • 扩展现有回归测试及 PR 工作流路径触发,保持现有目标skip及镜像别名映射。

Validation

  • 原回归路径先复现拒绝证明元数据的失败,再修复通过。
  • 真实 RustFS 内容复制至本地OCI目录:两个原平台/子manifest摘要一致,17个可达blob大小和SHA256一致,没有复制证明描述符。
  • 合成多架构OCI目录:amd64、arm/v7、riscv64及全部7个blob保持;singlemanifest原摘要及2个blob保持。
  • 独立只读复核上述终态及所有可达blob,未发现已证实的阻塞问题。
  • Shell语法、actionlint、现有工作流安全检查、diff-check通过。
bash scripts/tests/mirror-storage-images-test.sh
bash -n scripts/mirror-runtime-images.sh scripts/tests/mirror-storage-images-test.sh
actionlint .github/workflows/publish-images.yml .github/workflows/pr-scripts.yml
bash scripts/tests/workflow-security-test.sh
git diff --check

Risk

  • 仅改变CI依赖镜像复制,不改业务代码、数据库、存储数据或正式站点。
  • 镜像库不接受的证明元数据不会复制到目标仓库;运行镜像原始内容摘要不变,但过滤后的目标顶层index摘要与原始含证明index不同。上游证明仍可从原始源读取,不将其冒称镜像已保存。
  • 固定OCI工具的新CI依赖仅用于镜像同步,官方二进制URL及SHA256均固定;不引入业务生产依赖,不通过降级旧Buildx规避问题。
  • 初期仅mock通过、实际Buildx dry-run仍重新带入证明的方案已被否决;首次OCI验证的网络TLS EOF及编辑运行脚本造成的harness失败均已保留并由冻结脚本终态验证替代。
  • 本地测试没有写任何远端镜像仓库;真实阿里云写入需合并后运行正式工作流验收,不能凭本地OCI目录通过冒充恢复。

Notes

Signed-off-by: XiaoSeS <87064762+XiaoSeS@users.noreply.github.com>

@utafrali utafrali left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix is correct and well-targeted: replacing Buildx (which aggressively re-discovers referrers) with a pinned regctl that can copy images with referrers disabled solves the documented 403 from Aliyun. The Python-based descriptor filtering is careful and the test coverage is good. The main practical concern is the hardcoded amd64 binary URL, which will break silently on ARM runners.

- name: Install pinned registry copy tool
if: ${{ steps.mirror.outputs.enabled == 'true' }}
uses: docker/setup-buildx-action@v3
run: |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The binary URL hardcodes regctl-linux-amd64. If this job ever runs on a non-amd64 runner (GitHub now offers arm64 hosted runners), this step will silently download the wrong binary and the SHA256 check will catch it too late, failing the entire job with a confusing checksum error.

Since the SHA256 is arch-specific anyway, the simplest fix is to add a comment making the amd64 assumption explicit and document it near the runs-on: key for this job:

# This step pins the amd64 binary; the job must run on an amd64 runner.

or detect the arch dynamically with separate per-arch checksums.

echo 'FAIL: inspection failure was swallowed' >&2; exit 1
else [[ $? == "$expected" ]]; fi
else
if CREATE_EXIT="$expected" TARGET_EXISTS=false bash "$REPO_ROOT/scripts/mirror-runtime-images.sh" "$work/images" >"$work/output" 2>&1; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The else [[ $? == "$expected" ]]; fi construct does not actually fail the test if the exit code is wrong. The [[ ]] expression returns false, but the test script has no set -e and nothing forces a failure. The assertion is effectively dead.

Change to:

else
  actual=$?
  [[ "$actual" == "$expected" ]] || { echo "FAIL: expected exit $expected, got $actual" >&2; exit 1; }
fi

This also applies to the identical pattern in the create branch a few lines below.


- name: Set up Docker Buildx
- name: Install pinned registry copy tool
if: ${{ steps.mirror.outputs.enabled == 'true' }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The curl call has no --max-time flag. A stalled connection (not a refused connection, which --retry would catch) will hang until GitHub's 6-hour job timeout. Add --max-time 60 to bound the download time.

# Buildx also discovers and copies referrers even with explicit child sources.
# regctl can copy the original image blobs without adding those artifacts.
if [[ ${#source_refs[@]} -eq 1 && "${source_refs[0]}" == "$source" ]]; then
regctl image copy --referrers=false "$source" "${target_prefix}/${target}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --referrers=false flag is passed to regctl index create. This flag is well-documented for regctl image copy, but it is worth confirming the flag is also accepted by index create in v0.11.7. If it is silently ignored or unrecognized, multi-arch images would still have referrers propagated, reintroducing the 403. A quick regctl index create --help | grep referrer in the CI log would confirm it.

@XiaoSeS
XiaoSeS merged commit e7e38d7 into main Oct 25, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants