Skip to content

MVP: flag unresolved external dependencies during migration analysis #41

Description

@AlexDeMichieli

Why

During the Siemens Building X Core migration, the migrator translated visible CI syntax well, but some pipelines could still produce syntactically valid yet non-functional GitHub Actions workflows.

The blockers were not unique to GitLab:

  • Missing or deeply nested templates and inheritance: external templates, shared libraries, or inherited configuration can define jobs, variables, conditions, artifacts, and deployment behavior that are not visible in the analyzed files. Real pipelines expanded from a few hundred lines to several thousand once nested templates were resolved, so depth itself is a signal.
  • Opaque execution logic: jobs may delegate important behavior to custom container images, plugins, tasks, or binaries whose internals are unavailable.
  • Build-time or runtime external fetching: images or jobs may clone or download additional repositories, binaries, or scripts while running, adding dependencies that never appear in the CI file.
  • Source-platform coupling: referenced logic may depend on predefined variables, credentials, commands, or REST/GraphQL APIs from the source CI platform.
  • Absent scripts: pipelines may invoke repository scripts, Make targets, or other files that were not provided to the agent.
  • Technical debt and obsolete steps: pipelines accumulate deprecated steps, commented-out blocks, redundant stages, and unused secrets. Automation cannot tell which steps are still required, so it may faithfully carry forward logic that should be dropped.
  • Validation limits: syntax validation cannot prove that unseen runtime behavior was preserved.

The tool does not need to recursively resolve, fix, or prune these dependencies. It should detect likely boundaries, raise clear concerns, and avoid overstating completeness.

Proposed MVP

Add a platform-agnostic external dependency preflight to the shared Analyze phase. Using only provided/local files, the agent looks for:

  • referenced templates, includes, shared libraries, or inherited configuration whose targets are unavailable (or are nested unusually deep)
  • custom images, plugins, tasks, or binaries whose behavior is not visible
  • build-time or runtime fetching of external repositories, binaries, or scripts
  • referenced scripts, build files, or commands whose source is absent
  • visible source-platform variables, credentials, commands, or API calls that require adaptation for GitHub Actions
  • steps that appear deprecated, commented-out, or redundant, and secrets that may no longer be needed

Platform-specific skills may provide detection cues-for example GitLab include/extends, Jenkins shared libraries, Azure DevOps templates/tasks, CircleCI orbs, or Bitbucket pipes-but the behavior and report format remain in the shared migration core.

For each concern, add one row to the migration report:

Reference Concern Impact Action needed
Shared pipeline template Source not provided / deeply nested High Provide or review the template manually
Custom build image/plugin Internal behavior is opaque High Confirm required commands, variables, and credentials
Deployment script Uses source-platform integration High Adapt the integration before cutover
Legacy stage Possibly deprecated/unused Low Confirm with owners whether it is still needed

Then:

  1. Convert only behavior that is visible.
  2. Put unresolved dependencies and manual actions in the migration report, not placeholder workflow code. Leave behavior that cannot be safely translated unconverted; do not invent replacement commands or add TODO(migration) markers that the quality gate rejects.
  3. Ask for a missing local file when it is required to understand the pipeline.
  4. For remote, cross-repository, opaque, or possibly-obsolete dependencies, record the concern and defer the judgement to a human. Continue translating visible, understood behavior, but do not present affected workflows as ready to deploy while required behavior is unresolved.
  5. Report Migration incomplete: unresolved dependencies when an execution-critical concern remains. Use Migration complete with documented caveats only for non-critical concerns, and Migration complete only when no concerns remain.
  6. Expand only templates/configuration that were provided and are visible; report absent content rather than inferring it.

Minimal implementation

Keep the first iteration instruction-driven:

  • define the generic preflight behavior and result wording in the shared migration workflow/guardrails and CLI migration-core skill
  • add a small generic Dependency Preflight section to migration reports
  • add short source-platform cue lists to existing platform mapping skills only where needed

No parser, manifest, dependency graph, resolver, API integration, or hook enforcement is required for this MVP.

The current quality gate rejects TODO/FIXME placeholders, so this proposal reports gaps without adding those markers to workflows. Align implementation with #53: maintain both active routes during transition, but do not create another permanent copy.

#42 will need a separately agreed structured report format. This instruction/report MVP does not yet promise that format; agree that extension before starting graph implementation.

Trust boundaries

  • Never execute analyzed scripts or binaries.
  • Never pull or run referenced container images.
  • Never fetch remote or cross-repository content.
  • Record secret/variable names only, never values.
  • Never invent behavior for unavailable content.

Acceptance criteria

  • Shared Analyze guidance detects unavailable external dependencies for every supported source CI platform.
  • Platform skills provide cues without implementing different preflight behavior or output formats.
  • Reports list each concern, its likely impact, and a clear manual action.
  • Unresolved behavior is recorded in the report and left unconverted, not replaced with fabricated commands or workflow placeholders. Existing quality checks are not weakened to accept incomplete output.
  • Missing required local files cause the agent to ask for those files.
  • Remote/cross-repository/opaque concerns are reported without fetching or executing them.
  • Possibly deprecated or redundant steps are flagged for human review, not removed automatically.
  • Execution-critical concerns produce Migration incomplete: unresolved dependencies and do not result in a deploy-ready completion claim.
  • Guidance distinguishes syntax validation from behavioral completeness.

Non-goals

  • Recursively resolving or migrating subdependencies
  • Building a dependency graph or resolver engine
  • Fetching remote content or inspecting container filesystems
  • Executing source scripts, images, plugins, or tasks
  • Deciding whether steps or secrets are still needed, or automatically pruning obsolete steps (flag for human review only)
  • Guaranteeing behavioral equivalence through static analysis
  • Delivering a full preflight framework in the first iteration

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

agentsNew and udpated AgentsautomationNew and udpated GitHub Actions workflows and other automation scriptsknowledgebaseNew and updated knowledge articles

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions