Skip to content

feat(web): add DataFinder adapter for configuration-only analytics (#989, part 3) - #1050

Closed
xiaoyaner0201 wants to merge 10 commits into
iflytek:mainfrom
xiaoyaner0201:feat/datafinder-analytics-adapter-989
Closed

xiaoyaner0201 wants to merge 10 commits into
iflytek:mainfrom
xiaoyaner0201:feat/datafinder-analytics-adapter-989

Conversation

@xiaoyaner0201

@xiaoyaner0201 xiaoyaner0201 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Refs #989。基于 #1037,新增第二个注册适配器:DataFinder(火山引擎增长分析)。

依赖 #1037:本分支叠在 #1037 之上,#1037 合并前 diff 会包含它的提交。只需看最后 4 个提交(987068cc 起)。#1037 合并后我会 rebase,再转为 ready。

#1010 讨论时维护者同意“其他平台以后按需扩展”。我们自己部署 SkillHub,用的是私有化 DataFinder,所以先补这个平台。

  • 只改配置:adapter: "datafinder",adapterConfig 只接受数字 appId 和可选的 product(默认 skillhub),upstream 填采集域名。其他字段全部拒绝,包括 App Key,防止误把服务端密钥放进前端。
  • 不引入官方 SDK:官方 JS SDK 约 250 KB,而且不是开源许可。适配器直接用 fetch(keepalive) 发 DataFinder Web 上报的最小 /list 负载:一个 batch、一个事件。不批量等待,不持久化,不重试,每次最长 3 秒。
  • 事件内容:事件名就是原有动作 ID(例如 skill.download.request),params 只含原生 event。公共属性只有 product 和 module:module 是动作所在页面的路由模板(例如 /space/$namespace/$slug),不含真实 namespace、slug、query 或 hash,未匹配的页面记为 other。
  • 匿名标识:生成随机 UUID v4 作为 user_unique_id/web_id,存在 localStorage,用于按浏览器统计 UV。不绑定 SkillHub 账号。纯 HTTP 内网部署没有 crypto.randomUUID,改用 getRandomValues 生成。存储不可用时只在当前页用内存 ID。文档写明这不等于无身份数据,由部署方评估。
  • 成功判定:只认 e:0 且 sc:1;HTTP 200 不算成功。
  • 代理:只放行 POST /_skillhub-analytics/list,Umami 的端点返回 404。沿用 TLS 校验,并清除 Cookie、Authorization、Referer。保留浏览器 User-Agent,因为平台会拒收非浏览器 UA。
  • 通用层改动:bootstrap 提供一个与平台无关的 currentPageType getter,依次经 loader、controller 传给 factory。Umami 不使用它。通用配置和 controller 不认识 DataFinder 的字段。awk 校验器增加数字值和按适配器分支。
  • 文档:接入指南新增第 2.1 节,说明配置、Helm、核对步骤、排障表(UA 拒收、sc:0、502)和隐私边界;英文契约和 OpenSpec PR-3 任务同步更新。

Validation

head cfa48c9f:

corepack pnpm@10.33.0 -C web test        # 270 文件 / 1,431 项通过(#1037 为 1,380)
corepack pnpm@10.33.0 -C web typecheck   # 通过
corepack pnpm@10.33.0 -C web lint        # 通过
corepack pnpm@10.33.0 -C web build       # 通过
node web/scripts/analytics-manifest.mjs  # 通过
python3 scripts/tests/web-analytics-test.py          # 通过(含 DataFinder 校验与 /list 代理片段)
bash scripts/tests/validate-release-config-test.sh   # 通过
bash scripts/tests/web-base-path-routing-test.sh     # 通过
helm lint --strict + configuration-contracts.sh      # 通过(helm 3.19.0 容器,仅模板检查)

另外在本地用 mock 采集端做了浏览器集成,覆盖根路径和子路径。

还没覆盖: 真实 DataFinder 平台的端到端验收还没跑,会在转 ready 前补上,并贴出结果(e:0/sc:1 和平台事件页)。真实 Kubernetes 集群也没有验证。

Risk

  • 默认关闭;不选 datafinder 时,Umami 和其他行为不变。
  • 新增 localStorage 键 skillhub.analytics.datafinder.browser-id,只在启用 DataFinder 且已授权时写入。
  • 回滚:清空配置并重新创建 Web。
English

Refs #989. Adds DataFinder (Volcengine growth analytics) as the second registered adapter, stacked on #1037; review only the last four commits (from 987068cc). I will rebase and mark ready once #1037 merges.

  • Config-only: adapterConfig accepts a numeric appId and optional product; every other key (including App Keys) is rejected.
  • No official SDK: a bounded fetch(keepalive) sends the minimal DataFinder Web /list batch. One event per request, no queue or retry, 3 s cap. Success requires e:0 and sc:1.
  • Payload: event name is the existing intent ID; params holds only the native event; public properties are product and module (route template, never real path values).
  • Anonymous browser UUID in localStorage as user_unique_id/web_id, with a getRandomValues fallback for plain-HTTP origins. No account binding.
  • Proxy exposes only POST /_skillhub-analytics/list, keeps TLS verification and header stripping, preserves the browser User-Agent.
  • Generic layer gains an optional vendor-neutral currentPageType context.

Validation: 1,431 web tests, typecheck, lint, build, manifest, script tests and full Helm template gates pass. A local mock-collector browser run was done. Real-platform acceptance is still pending and will be posted before ready; no real-cluster Helm install.

Move the passive helper and Umami adapter into one product source, retain the existing assertions, and add consent-aware lazy lifecycle control.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Safely transport bounded JSON using existing image tools, generate isolated exact proxy routes with verified HTTPS, and wire Compose and Helm with closed defaults.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Replace manual injection instructions with Compose, Docker and Helm configuration, explicit platform initialization, consent controls, privacy boundaries and verified task status.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Clarify the one-time initialization sequence verified during the guide walkthrough.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Helm --set-string parses {...} as a list, so the enabled-analytics
contract case failed schema validation in CI. Use a values file, the
same way the operator guide documents the setting.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Move analytics values into a Web-only ConfigMap. The server checksum hashes
the shared ConfigMap, so analytics-only changes used to replace server pods.
Contract tests render before/after values and assert that analytics-only
changes roll only Web while backend changes still roll the server.

Docs: Compose restart does not re-read .env.release, old tabs keep the old
config, a valid but unknown Umami website returns 400 and stores nothing,
and new adapters also need deploy-side validation, proxy endpoints and
tests. OpenSpec now separates executed Helm template checks from the
real-cluster verification that has not been run.

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
crypto.randomUUID only exists in secure contexts and newer browsers, so
fresh browsers on plain-HTTP intranet deployments dropped every event.
Fall back to a getRandomValues v4 UUID with the same format.

Refs iflytek#989

Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com>
@xiaoyaner0201

Copy link
Copy Markdown
Contributor Author

按讨论合入 #1037 一起评审:DataFinder adapter 与 Umami 共用同一套配置化转换层,和一键部署(后续 PR)相互独立。提交已原样并入 #1037(head ae62cb0d)。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant