Script steps exit with code 2 before script runs on v1.6.0

[bryanbaer]

Summary

Script-based steps now fail immediately with exit code 2 before the user script runs when Tekton Pipelines v1.6.0 executes in our cluster. This happens even for a one-step TaskRun using BusyBox; kubectl logs (and crictl logs) show no output. The only indication of failure is the TaskRun condition message "step-say" exited with code 2: Error. The pod events show the step container being started and then killed a second later.

Steps to Reproduce

1. Have Tekton Pipelines v1.6.0 installed (controller image gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/controller:v1.6.0@sha256:255ea8...). Our namespace is labeled with Pod Security enforce=baseline, but the failure reproduces even if we remove those labels.
2. Apply the following TaskRun in the Tekton namespace:

apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
  name: simple-echo-tr
  namespace: tekton-pipelines
spec:
  serviceAccountName: default
  taskSpec:
    steps:
      - name: say
        image: 10.34.0.202:5000/mirror/busybox:1.35
        script: |
          #!/bin/sh
          echo "hello from taskrun"
          sleep 1

3. Wait a few seconds and inspect the pod / TaskRun status:

$ kubectl describe taskrun simple-echo-tr -n tekton-pipelines
...
  Conditions:
    Message: "step-say" exited with code 2: Error
...
  Steps:
    Name: say
    Terminated:
      Exit Code: 2
      Reason: Error

4. kubectl logs simple-echo-tr-pod -c step-say returns nothing. crictl logs against the container ID is also empty. The /tekton/termination file in the pod is empty as well, so there is no error output to capture.

This also affects PipelineRuns: even a single-task PipelineRun that echoes "hello" fails the same way.

Expected behavior

The step script should run (printing the echo and sleeping) and succeed. This same manifest worked earlier today before we started seeing the regression.

Actual behavior

The step container exits almost immediately with exit code 2. No logs or error message are produced. The TaskRun status just reports the exit code. Because no script runs, any Task/Pipeline that uses script: steps fails before doing work.

Additional information

• Tekton Pipelines version: v1.6.0 (controller/webhook images from the official release YAML)
• Kubernetes version: v1.30.14 (server), containerd v2.1.5
• Feature flags (relevant ones):
  • set-security-context: "false"
  • disable-creds-init: "false"
  • results-from: termination-message
  • running-in-environment-with-injected-sidecars: "true"
• Namespace labels: pod-security.kubernetes.io/enforce=baseline, .../warn=restricted, .../audit=restricted
• Container runtime shows the entrypoint command as [ "/tekton/bin/entrypoint", "-wait_file", "/tekton/downward/ready", ..., "-entrypoint", "/tekton/scripts/script-0-xxxx", "--" ] — the container exits before invoking the script.
• We also tried disabling creds-init (setting disable-creds-init: true and restarting the controller/webhook) and re-running the TaskRun; the behavior was unchanged.

Let me know what other diagnostics would be helpful. Happy to provide the full pod YAML or run a custom entrypoint build if needed.

[vdemeester]

Hello @bryanbaer, I have a couple of questions

1. What was the previous working version? When did you upgrade from which version to v1.6.0? This helps narrow down which commits introduced the regression.
2. On which architecture are you running tektoncd/pipeline ?
3. Are there any controller/webhook logs? Check if the controller or webhook pods show any errors around the time of TaskRun creation: kubectl logs -n tekton-pipelines deployment/tekton-pipelines-controller --tail=100
4. What does the entrypoint command show exactly? (aka without the ...)
5. Can you inspect the failed container? Use crictl inspect to see if there are any additional error details or exit reasons.

(controller image gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/controller:v1.6.0@sha256:255ea8...).

Are you sure about this one ? @afrittoli shouldn't the images for 1.6.0 be only available from ghcr.io ?

[bryanbaer]

1) Previous working version / upgrade timing We were running Pipelines v1.5.3 (applied from the previous/v1.5.3/release.yaml bundle) from late October until 21 Nov. On 21 Nov we upgraded directly to v1.6.0 by applying https://storage.googleapis.com/tekton-releases/pipeline/previous/v1.6.0/release.yaml . The regression started immediately after that upgrade; the exact same TaskRuns succeeded a few minutes earlier on v1.5.3. 2) Architecture is linux/amd64 3) Controller / webhook logs $ kubectl logs -n tekton-pipelines deploy/tekton-pipelines-controller

--tail=100

{"level":"info","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:466","msg":"Reconcile

succeeded","key":"tekton-pipelines/simple-echo-tr"}

{"level":"info","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:466","msg":"Reconcile

succeeded","key":"tekton-pipelines/git-clone-debug-tr"}

(no errors/warnings around the failure timestamp) $ kubectl logs -n tekton-pipelines deploy/tekton-pipelines-webhook

--tail=100

(only readiness/health logs, no errors) So the controller/webhook never emitted an error; it simply recorded the failed exit code coming back from the step container.Entrypoint commandFrom crictl inspect on a failed step container: 4) Entrypoint command (full args) /tekton/bin/entrypoint \ -wait_file /tekton/downward/ready \ -wait_file_content \ -post_file /tekton/tools/step-post-file \ -termination_path /tekton/termination \ -step_metadata_dir /tekton/steps \ -results /tekton/results/result \ -step_results /tekton/results/step-result \ -entrypoint /tekton/scripts/script-0-LL8vX \

-- /tekton/scripts/script-0-LL8vX So it’s the stock /tekton/bin/entrypoint binary invoking the generated shell script; the container never reaches the script because it exits with code 2 inside entrypoint. 5) crictl inspect $ sudo crictl inspect <container-id> | jq '.info.runtimeSpec.process' { "args": [ "/tekton/bin/entrypoint", "-wait_file","/tekton/downward/ready", "...", "-entrypoint","/tekton/scripts/script-0-LL8vX", "--","/tekton/scripts/script-0-LL8vX" ], "env": [...], "cwd": "/workspace/output" } $ sudo crictl inspect <container-id> | jq '.status' { "id": "...", "metadata": {...}, "state": "CONTAINER_EXITED", "exitCode": 2, "reason": "Error", "finishedAt": "2023-11-21T18:42:17.214372123Z", "runtimeHandler": "" } No additional error message is recorded; the termination message file is empty as well. Image registry question You’re right to ask—those SHA references came straight from the official 1.6.0 YAML in the Google Storage bucket, which still points at gcr.io/tekton-releases/... images: image: ***@***.***:255ea8. .. image: ***@***.***:ee4b25. .. We didn’t pull anything custom; we simply applied the published release manifest. If GHCR-only images are the new standard, that bucket probably needs to be updated, because that’s what we used.Let me know if you need any other artifacts (full TaskRun YAML, crictl logs, etc.) and I’ll ship them over. Thanks for looking into this! It's been driving me crazy!

-Bryan

On Mon, Nov 24, 2025 at 5:13 AM Vincent Demeester ***@***.***> wrote: vdemeester left a comment (tektoncd/pipeline#9162) Hello @bryanbaer, I have a couple of questions What was the previous working version? When did you upgrade from which

version to v1.6.0? This helps narrow down which commits introduced the regression.

On which architecture are you running tektoncd/pipeline ? Are there any controller/webhook logs? Check if the controller or webhook

pods show any errors around the time of TaskRun creation: kubectl logs -n tekton-pipelines deployment/tekton-pipelines-controller --tail=100

What does the entrypoint command show exactly? (aka without the ...) Can you inspect the failed container? Use crictl inspect to see if there

are any additional error details or exit reasons.

(controller image

***@***.***:255ea8... ).

Are you sure about this one ? @afrittoli shouldn't the images for 1.6.0

be only available from ghcr.io ?

…

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned. Message ID: ***@***.***>

--