Skip to content

feat(web): configuration-only optional frontend analytics - #1037

Open
xiaoyaner0201 wants to merge 6 commits into
iflytek:mainfrom
xiaoyaner0201:feat/configurable-frontend-analytics-989
Open

xiaoyaner0201 wants to merge 6 commits into
iflytek:mainfrom
xiaoyaner0201:feat/configurable-frontend-analytics-989

Conversation

@xiaoyaner0201

@xiaoyaner0201 xiaoyaner0201 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Refs #989(两步交付的第二步。合并并验证后,可以关闭 #989)

按 #1010 合并时的约定和 OpenSpec openspec/changes/add-configurable-frontend-analytics/ 里的 PR-2 部分,把前端动作统计做成「改配置就能开」的可选功能。

  • 只改配置:Docker/Compose 和 Helm 用户填两项配置就能开启:SKILLHUB_WEB_ANALYTICS_CONFIG(JSON:enabled、adapter、consent、rules、adapterConfig)和 SKILLHUB_WEB_ANALYTICS_UPSTREAM(统计平台地址,只在部署侧使用,不会下发到浏览器)。不需要改 JS、复制文件或手写 Nginx。
  • 默认关闭、按需加载:页面挂载完成后才读取统计配置。没配置、enabled:false、consent:denied 或配置写错时,不加载采集模块和 SDK,也不发请求;页面启动不等待统计平台。
  • 故障隔离:配置写错只关闭统计,容器日志只提示出错的字段名,不回显原值;其他 runtime-config 字段保持不变。统计平台域名在请求时才解析,解析失败或平台停机都不影响 Web 启动和业务。
  • 单一产品源码:helper 和 Umami adapter 迁到 web/src/features/analytics/(通用类型、controller、loader、静态注册表、adapters/umami)。feat(web): add passive semantic analytics attributes across frontend workflows #1010 的两组测试一起迁入,断言没有改动。通用部分不认识 Umami 的字段,以后加平台只需新增一个注册的 adapter。
  • 同源代理:Web 容器启动时,只在开启后生成 /_skillhub-analytics/ 代理。只放行 Umami 的 script.js 和 api/send,其他路径都返回 404,不暴露管理界面。HTTPS 上游会校验证书并带正确的 SNI;代理会去掉 Cookie、Authorization、Referer。保留前缀已同步到 20-base-path.sh、base-path-config.ts、validate-release-config.sh 和 Helm 校验。子路径部署可用。
  • 没有新增依赖:没按设计稿引入 jq。用镜像自带的 awk 校验 JSON(限制 64 KiB、32 层、拒绝重复键),再用 base64 写进 runtime-config.js,浏览器端按数据解析。代价是多了一个约 90 行、有测试覆盖的小解析器。
  • 部署接线:compose.release.yml、.env.release.example 和 Helm 的 values、schema、ConfigMap、Deployment 都已透传,默认关闭。新增可选的 compose.analytics-umami.yml,用来自托管 Umami 和它自己的 Postgres:必须显式提供密码,管理端口只绑本机,不额外增加 Nginx。
  • 文档:docs/frontend-analytics-integration.md 改写成只改配置的中文接入指南,内容包括已有 Umami、自托管组合、动态授权、验收、排障、升级和关闭;docs/frontend-analytics.md 更新为英文契约。docs/examples/analytics/ 只保留一个指路说明。OpenSpec tasks.md 只勾选了本 PR 实际完成的项目。

Validation

基于 6c276d38(当前 main),head f8ab71e2。

corepack pnpm@10.33.0 -C web test        # 269 文件 / 1,380 项通过(基线 1,357)
corepack pnpm@10.33.0 -C web typecheck   # 通过
corepack pnpm@10.33.0 -C web lint        # 通过
corepack pnpm@10.33.0 -C web build       # 通过
node web/scripts/analytics-manifest.mjs  # 通过,清单内容逐字节不变
python3 scripts/tests/web-analytics-test.py          # 10 项通过(启动脚本、配置序列化、恶意 JSON、代理片段)
bash scripts/tests/web-base-path-routing-test.sh     # 通过
bash scripts/tests/validate-release-config-test.sh   # 通过
bash scripts/tests/web-base-path-nginx-smoke-test.sh # 通过(真实 nginx 容器)
bash scripts/tests/nginx-forwarded-proto-test.sh     # 通过;结尾的 `ip_args[@]: unbound variable` 在 main 上同样出现,与本 PR 无关
helm lint --strict charts/skillhub -f charts/skillhub/tests/test-values.yaml   # 通过(helm 3.19.0 容器)
bash charts/skillhub/tests/configuration-contracts.sh                        # 通过;含「只改统计仅 Web 更新、改后端配置后端仍更新」的 checksum 对照
docker compose --env-file .env.release.example -f compose.release.yml config
docker compose --env-file .env.release.example -f compose.release.yml -f compose.analytics-umami.yml config   # 需要提供密码,不提供时会按设计报错

另外做了本地真实运行验收:用本 PR 构建的前端、仓库里的 Nginx 模板和启动脚本,配合 Umami 3.4.1 与 Postgres,在真实 Chrome 中完成 26 项检查,Umami 实际入库 9 条事件。检查覆盖:

  • 默认零模块、零请求
  • 开启后页面挂载完成才加载
  • 实际出站请求只包含白名单字段
  • 配置写错、adapter 未知、上游地址非法时,只关闭统计
  • 域名无法解析、Umami 停机时,业务不受影响
  • 未注册的代理路径返回 404
  • 调用 stop() 后不再发送
  • /skillhub/ 子路径部署
  • 5 MiB 上传仍能到达 API

SkillHub 后端 API 是合成响应,不能作为后端业务的端到端证明。

还没覆盖的部分:

  • 镜像离线构建时,Corepack 需要联网,所以没有在离线环境完成整镜像构建。真实运行验收使用同样的 dist、模板和启动脚本组装容器完成。
  • OpenSpec 任务 6.3 中完整的 release Compose S3/auth 冒烟,以及 6.1、6.4 的评审和验收环节,都留给维护者。

Risk

  • User-facing impact: 默认关闭,未配置时不加载采集代码、不发请求,行为和现在一致。
  • Deployment or migration impact: Web 镜像新增一个启动脚本和一个 awk 文件,Compose 和 Helm 新增可选的环境变量;没有数据库或后端变更,也没有新增 npm、apk 依赖。/_skillhub-analytics/ 成为保留前缀,SKILLHUB_WEB_BASE_PATH 不能以它开头。修改配置后需要重新创建 Web(Compose 用 up -d --force-recreate web,restart 不会重新注入环境变量),新打开的页面才会生效;已经打开的旧标签页不会因关闭配置立即停止,需要刷新或关闭。Helm 的统计值放在 Web 专用 ConfigMap,只改统计配置时仅 Web 滚动更新,后端 Pod 不替换。
  • Rollback approach: 清空 SKILLHUB_WEB_ANALYTICS_CONFIG 并重新创建 Web,新页面即完全关闭;回退镜像版本也可以。

Notes

  • Related issue: [Feature] 支持厂商中立的前端业务语义属性 #989
  • 新平台适配除了注册前端 adapter,还要在 Web 启动脚本加该平台的部署侧配置校验和精确代理端点,并补测试。
  • 配置校验只检查格式:websiteId 是合法 UUID 但 Umami 中没有该网站时,api/send 返回 400 Website not found,事件不入库。排障表已补充。
  • Follow-up work: 其他平台的 adapter 按需另提;一键部署统计平台另开 PR;本 PR 不包含后端业务事件([Feature] RFC:默认关闭的厂商中立后端业务事件出口 / Opt-in backend business-event export #1009)、同意弹窗、重试队列或管理后台。
  • Docs or operator runbooks updated when behavior changed: 已更新 docs/frontend-analytics-integration.md、docs/frontend-analytics.md、docs/09-deployment.md 和 chart README 的入口链接。
English

Refs #989 (second step; #989 can be closed once this is merged and verified).

Implements the PR-2 part of openspec/changes/add-configurable-frontend-analytics/, as agreed when #1010 was merged:

  • Configuration only: set SKILLHUB_WEB_ANALYTICS_CONFIG (JSON: enabled, adapter, consent, rules, adapterConfig) and SKILLHUB_WEB_ANALYTICS_UPSTREAM (deploy-side origin, never sent to the browser) through Docker/Compose or Helm. No JS, file copying or hand-written Nginx.
  • Default off, loaded on demand: analytics config is read after the app mounts. Absent config, enabled:false, consent:denied or invalid config means no analytics chunk, no SDK and no request. App startup never waits on the collector.
  • Failure isolation: invalid config only disables analytics, with field-level diagnostics that never echo raw values. Other runtime-config keys stay intact. The collector host is resolved per request, so DNS failure or collector downtime does not affect Web startup or business flows.
  • Single product source: the helper and Umami adapter moved to web/src/features/analytics/ (types, controller, loader, static registry, adapters/umami). Both feat(web): add passive semantic analytics attributes across frontend workflows #1010 test suites moved with their assertions unchanged. Generic code has no Umami fields, and new platforms are added as registered adapters.
  • Same-origin proxy: generated at startup only when enabled. Only Umami script.js and api/send are proxied under /_skillhub-analytics/; all other paths return 404. HTTPS upstreams are verified with correct SNI, and Cookie, Authorization and Referer are stripped. The reserved prefix is synced across the base-path checks and Helm validation, and sub-path deployment works.
  • No new dependency: instead of jq, the image's own awk validates the JSON (64 KiB, 32 levels, duplicate keys rejected). Base64 carries it into runtime-config.js, where the browser parses it as data. The trade-off is a small, tested parser of about 90 lines.
  • Deployment wiring: Compose, .env.release.example and Helm values, schema, ConfigMap and Deployment pass the config through, with analytics off by default. The optional compose.analytics-umami.yml overlay self-hosts Umami and its own Postgres, requires explicit secrets, binds the admin port to loopback only and adds no extra Nginx.
  • Docs: a Chinese configuration-only guide and an updated English contract. docs/examples/analytics/ is now a pointer only. OpenSpec ticks cover only the work done in this PR.

Validation at head f8ab71e2 on base 6c276d38:

  • Frontend: 1,380 tests (baseline 1,357); typecheck, lint, build and manifest (byte-identical) all pass.
  • Script tests: 10 new startup/serialization tests; base-path routing, release-config validation, nginx sub-path smoke and forwarded-proto all pass. The trailing unbound variable from forwarded-proto also appears on main.
  • Compose: default and overlay render. The overlay requires secrets by design.
  • Helm: helm lint --strict and configuration-contracts.sh pass (helm 3.19.0 container). Analytics lives in a Web-only ConfigMap; the contract test asserts analytics-only changes roll only Web and backend changes still roll the server. These are template checks; no real-cluster install was run.

A local real-runtime run (this PR's dist with the repo's Nginx template and entrypoints, plus Umami 3.4.1, Postgres and real Chrome) passed 26 checks, with 9 events stored. Covered: default zero requests, enabled loading, allowlisted payload, invalid config, DNS failure and collector downtime isolation, 404 for unregistered paths, stop(), /skillhub/ sub-path and a 5 MiB upload. SkillHub API responses were synthetic.

Not run locally:

  • An offline full image build, because Corepack needs the network.
  • The full release Compose S3/auth smoke, left to maintainers.

Rollback: clear SKILLHUB_WEB_ANALYTICS_CONFIG and recreate Web (docker compose restart does not re-read the env file). Old tabs keep collecting until reloaded or closed.

Notes: new platforms need a registered adapter plus deploy-side validation, proxy endpoints and tests. A valid UUID for a website that does not exist in Umami gets 400 Website not found and stores nothing; passing config validation does not prove collection.

Move the passive helper and Umami adapter into one product source, retain the existing assertions, and add consent-aware lazy lifecycle control.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Safely transport bounded JSON using existing image tools, generate isolated exact proxy routes with verified HTTPS, and wire Compose and Helm with closed defaults.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Replace manual injection instructions with Compose, Docker and Helm configuration, explicit platform initialization, consent controls, privacy boundaries and verified task status.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Clarify the one-time initialization sequence verified during the guide walkthrough.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Helm --set-string parses {...} as a list, so the enabled-analytics
contract case failed schema validation in CI. Use a values file, the
same way the operator guide documents the setting.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
@XiaoSeS

XiaoSeS commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

我们基于当前提交 6cee2fe 做了一轮本地核验,整体方案可行:配置化接入、默认关闭、按需加载和适配器扩展的方向都符合预期。

本次使用源码构建的真实镜像、真实 SkillHub 后端和 Umami 验证了:

  • 根路径、子路径下的登录、上传下载和事件采集。
  • 实际事件入库、发送字段白名单、停止与恢复采集。
  • 错误配置、统计平台停机、DNS/证书失败时的业务隔离。
  • Nginx 端点和方法限制,以及敏感请求头过滤。

这些主要链路均通过,暂未发现阻断性的浏览器或代理运行问题。建议本 PR 再调整以下两部分:

1. 隔离 Helm 统计配置的影响

目前统计字段放在共享 ConfigMap 中,而后端 Pod 的 checksum 覆盖整份 ConfigMap。因此仅修改统计开关、规则、网站 UUID 或上游地址,也会触发后端 Pod 替换;部分存储配置使用 Recreate,影响会更明显。

我们已通过 Helm 模板渲染对照确认。建议将统计配置拆到 Web 专用 ConfigMap,或收窄后端 checksum 的输入范围,并补测试:只改统计配置时,仅 Web 更新;修改后端配置时,后端仍正常更新。无需为此大范围重构 Chart。

2. 对齐文档与验收记录

  • 修改 .env.release 后,应明确需要重新创建 Web;单纯 restart 不会重新注入环境变量。
  • 关闭配置只影响重新加载配置的页面,旧标签页不会立即停止;指南已有说明,PR 正文同步即可。
  • OpenSpec 中仍有 Helm“未验证”的旧描述,请按实际执行情况更新,并区分模板检查与真实集群验证。
  • 新平台适配除 adapter 外,还涉及部署侧配置校验、代理端点和测试,正文可简短补充。
  • 排查说明可补充:UUID 格式合法但网站不存在时,实测 Umami 返回 400 Website not found,不会入库;配置校验通过不代表采集成功。

一键部署增强可以另开 PR,不要求本次完成。

建议后续在现有 runtime.sh 中增加可选的统计平台部署模式,使用独立 Compose 项目、配置目录和数据卷,与 SkillHub 分别管理启停和升级。接入采用两步:

  1. 部署 Umami+数据库,修改初始密码、创建网站并取得 UUID。
  2. 填写 SkillHub 统计配置,重新创建 Web,开启采集。

已有统计平台的部署方跳过第一步。文档说明网络连通、密钥复用、升级和备份即可,不必为了“一键”增加自动创建网站的逻辑。

当前 PR 保留现有配置化接入和可选 Compose 方式即可。完成上述 Helm 隔离和文档调整后,我们再核验受影响部分。

@xiaoyaner0201

xiaoyaner0201 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor Author

感谢细致的核验,两部分都已按建议调整,新 head 是 f8ab71e2。

1. Helm 统计配置隔离

统计的两个值从共享 ConfigMap 移到 Web 专用的 <release>-web-analytics ConfigMap,后端 checksum 的输入不再包含它们,Chart 其他部分没有动。

configuration-contracts.sh 新增对照测试,渲染改动前后的 Deployment checksum:

  • 只改统计配置:Web 的 checksum 变化,后端和其他 Deployment 的 checksum 全部不变
  • 改后端配置(例如 bootstrapAdmin.displayName):后端 checksum 仍然变化

在旧模板上跑这组测试会失败;我也试过把统计值同时留在共享 ConfigMap,测试同样会拦下。helm lint --strict 和整套契约测试在 helm 3.19.0 容器里通过。

2. 文档与验收记录

  • 接入指南、英文契约和 .env.release.example 都写明:改 .env.release 后要 up -d --force-recreate web,restart 不会重新注入环境变量
  • PR 正文已同步:关闭配置后,旧标签页要刷新或关闭才会停止
  • OpenSpec 4.3、6.2 按实际执行更新:Helm 只做了模板检查(lint、契约测试),真实集群安装没有跑
  • 新平台适配:指南和正文补充说明,除了前端 adapter,还要改启动脚本里的部署侧校验、代理端点,并补测试
  • 排障表补充 400 Website not found:UUID 合法但 Umami 中没有这个网站时不会入库,配置校验通过不代表采集成功

一键部署按你们的建议另开 PR。辛苦再核验一下受影响的部分。

Move analytics values into a Web-only ConfigMap. The server checksum hashes
the shared ConfigMap, so analytics-only changes used to replace server pods.
Contract tests render before/after values and assert that analytics-only
changes roll only Web while backend changes still roll the server.

Docs: Compose restart does not re-read .env.release, old tabs keep the old
config, a valid but unknown Umami website returns 400 and stores nothing,
and new adapters also need deploy-side validation, proxy endpoints and
tests. OpenSpec now separates executed Helm template checks from the
real-cluster verification that has not been run.

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
@xiaoyaner0201
xiaoyaner0201 force-pushed the feat/configurable-frontend-analytics-989 branch from 47285e4 to f8ab71e Compare November 1, 2026 12:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants