Skip to content

[Bug]: Error starting container when using DIND in Kubernetes pod #11139

New issue
New issue
Open
Open
[Bug]: Error starting container when using DIND in Kubernetes pod#11139
Labels
type/bug

Description

@michaelhalllloyds
michaelhalllloyds
opened on Oct 31, 2025

Module

Core

Testcontainers version

1.21.3

Using the latest Testcontainers version?

Yes

Host OS

Ubuntu

Host Arch

x86

Docker version

Docker version 20.10.15, build fd82621

What happened?

I am trying to run a container in a SpringBootTest using TestContainers as part of a Jenkins pipeline that spins up a Kubernetes pod in GCP.

My build/test is running in a Gradle container (built on Ubuntu), and it has /var/run and /var/lib/docker mounted from a docker-dind container.

When the test runs, it seems to start normally and attempt to pull and run the container:

11:01:32  2025-10-31T11:01:31.915+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:31.915Z","severity":"INFO","message":"Testcontainers version: 1.21.3","process":{"thread":{"name":"Test worker"}}}
11:01:34  2025-10-31T11:01:34.609+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:34.609Z","severity":"INFO","message":"Found Docker environment with local Unix socket (unix:///var/run/docker.sock)","process":{"thread":{"name":"Test worker"}}}
11:01:34  2025-10-31T11:01:34.620+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:34.620Z","severity":"INFO","message":"Docker host IP address is localhost","process":{"thread":{"name":"Test worker"}}}
11:01:34  2025-10-31T11:01:34.641+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:34.640Z","severity":"INFO","message":"Connected to docker: \n  Server Version: 20.10.15\n  API Version: 1.41\n  Operating System: Alpine Linux v3.15 (containerized)\n  Total Memory: 32102 MB","process":{"thread":{"name":"Test worker"}}}
11:01:34  2025-10-31T11:01:34.718+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:34.718Z","severity":"WARN","message":"Image name testcontainers/ryuk:0.12.0 was substituted by configuration to europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0. This approach is deprecated and will be removed in the future","process":{"thread":{"name":"Test worker"}}}
11:01:34  2025-10-31T11:01:34.719+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:34.718Z","severity":"INFO","message":"Using europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0 as a substitute image for testcontainers/ryuk:0.12.0 (using image substitutor: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor'))","process":{"thread":{"name":"Test worker"}}}
11:01:35  2025-10-31T11:01:34.818+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:34.818Z","severity":"INFO","message":"Pulling docker image: europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0. Please be patient; this may take some time but only needs to be done once.","process":{"thread":{"name":"Test worker"}}}
11:01:37  2025-10-31T11:01:36.974+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:36.973Z","severity":"INFO","message":"Starting to pull image","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37  2025-10-31T11:01:37.016+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.016Z","severity":"INFO","message":"Pulling image layers:  0 pending,  0 downloaded,  0 extracted, (0 bytes/0 bytes)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37  2025-10-31T11:01:37.282+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.282Z","severity":"INFO","message":"Pulling image layers:  2 pending,  1 downloaded,  0 extracted, (43 KB/? MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37  2025-10-31T11:01:37.295+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.295Z","severity":"INFO","message":"Pulling image layers:  1 pending,  2 downloaded,  0 extracted, (70 KB/? MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37  2025-10-31T11:01:37.337+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.336Z","severity":"INFO","message":"Pulling image layers:  0 pending,  3 downloaded,  0 extracted, (93 KB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37  2025-10-31T11:01:37.463+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.462Z","severity":"INFO","message":"Pulling image layers:  0 pending,  3 downloaded,  1 extracted, (3 MB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37  2025-10-31T11:01:37.591+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.591Z","severity":"INFO","message":"Pulling image layers:  0 pending,  3 downloaded,  2 extracted, (4 MB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37  2025-10-31T11:01:37.722+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.722Z","severity":"INFO","message":"Pulling image layers:  0 pending,  3 downloaded,  3 extracted, (6 MB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:38  2025-10-31T11:01:37.739+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.739Z","severity":"INFO","message":"Image europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0 pull took PT2.920436798S","process":{"thread":{"name":"Test worker"}}}
11:01:38  2025-10-31T11:01:37.816+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:37.815Z","severity":"INFO","message":"Creating container for image: europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0","process":{"thread":{"name":"Test worker"}}}

then something breaks:

11:01:39  2025-10-31T11:01:39.553+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:39.553Z","severity":"INFO","message":"Container europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0 is starting: 3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2","process":{"thread":{"name":"Test worker"}}}
11:01:40  2025-10-31T11:01:40.311+0000 [DEBUG] [TestEventLogger]     {"timestamp":"2025-10-31T11:01:40.309Z","severity":"ERROR","message":"Could not start container","process":{"thread":{"name":"Test worker"}},"error":{"type":"com.github.dockerjava.api.exception.NotFoundException","message":"Status 404: {\"message\":\"No such container: 3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2\"}\n","stack_trace":"com.github.dockerjava.api.exception.NotFoundException: Status 404: {\"message\":\"No such container: 3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2\"}

On the DIND side:

ERROR 2025-10-31T11:01:39.658837495Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.658598613Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
ERROR 2025-10-31T11:01:39.658887882Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.658651357Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
ERROR 2025-10-31T11:01:39.658895372Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.658661442Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
ERROR 2025-10-31T11:01:39.659318815Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.659018953Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2 pid=415 runtime=io.containerd.runc.v2
ERROR 2025-10-31T11:01:39.813347912Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813119378Z" level=info msg="shim disconnected" id=3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2
ERROR 2025-10-31T11:01:39.813381129Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813166233Z" level=info msg="ignoring event" container=3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
ERROR 2025-10-31T11:01:39.813398233Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813207171Z" level=warning msg="cleaning up after shim disconnected" id=3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2 namespace=moby
ERROR 2025-10-31T11:01:39.813401862Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813222671Z" level=info msg="cleaning up dead shim"

Activity

  1. michaelhalllloyds
    added
    type/bug
    on Oct 31, 2025
  2. marktech0813
    added a commit that references this issue on Nov 11, 2025

    [Bug]: Error starting container when using DIND in Kubernetes pod tes…

    9104105

  3. michaelhalllloyds commented on Nov 24, 2025

    @michaelhalllloyds
    michaelhalllloyds
    on Nov 24, 2025
    Author

    Based on the above PR, it has been possible to touch /.dockerenv to make testcontainers happy with the mounted docker sock. I suspect this fix will make that step unnecessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions