Skip to content

Investigate musllinux-arm64 CI test hangs for Python 3.9, 3.10, 3.11 #36

Description

@korniltsev-grafanista

Description

In the linux-test CI matrix added in grafana/pyroscope-rs#364, the musllinux-arm64 variant hangs indefinitely for Python 3.9, 3.10, and 3.11, while 3.12 and 3.13 complete successfully (~37s each).

All four variants started at the same time. The three older versions never complete.

Observed behaviour

Job Status Duration
musllinux-arm64 - 3.9 stuck in_progress 15+ min
musllinux-arm64 - 3.10 stuck in_progress 15+ min
musllinux-arm64 - 3.11 stuck in_progress 15+ min
musllinux-arm64 - 3.12 completed ~37s
musllinux-arm64 - 3.13 completed ~37s

Example run: https://github.com/grafana/pyroscope-rs/actions/runs/22293868489

Possible causes

  • python:3.9-alpine, python:3.10-alpine, python:3.11-alpine images may not have an arm64 variant on Docker Hub, causing docker pull to hang or fail silently
  • py-spy may not support profiling on Alpine arm64 for older Python versions
  • The test watchdog (120s timeout) fires but the process doesn't exit cleanly inside the container

Current workaround

These three combinations are excluded from the matrix via exclude: in ci-ffi-python.yml until the root cause is identified.

Next steps

  • Check Docker Hub for python:3.9-alpine / 3.10-alpine / 3.11-alpine arm64 manifest availability
  • Run the docker pull manually on an arm64 runner to confirm
  • Check if py-spy works on Alpine arm64 for these Python versions

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions