Skip to content

fix: allow zero workers when launcher acts as worker#833

Open
immanuwell wants to merge 1 commit into
kubeflow:masterfrom
immanuwell:fix/launcher-as-worker-zero-workers
Open

fix: allow zero workers when launcher acts as worker#833
immanuwell wants to merge 1 commit into
kubeflow:masterfrom
immanuwell:fix/launcher-as-worker-zero-workers

Conversation

@immanuwell

Copy link
Copy Markdown
Member

small fix for a sneaky runLauncherAsWorker footgun.

if runLauncherAsWorker: true is used and the Worker spec omits replicas, defaulting sets worker replicas to 0. reconcile then validates the defaulted job, hits ValidationError, and bails out, so the launcher Job never shows up.

this patch allows Worker.replicas: 0 only for that mode. other zero worker cases still fail, so the old guardrail stays.

repro:

  1. apply this:
apiVersion: kubeflow.org/v2beta1
kind: MPIJob
metadata:
  name: repro
spec:
  runLauncherAsWorker: true
  runPolicy:
    cleanPodPolicy: Running
  mpiReplicaSpecs:
    Launcher:
      template:
        spec:
          containers:
          - name: main
            image: mpi-image
    Worker:
      template:
        spec:
          containers:
          - name: main
            image: mpi-image
  1. on current master, the job gets a ValidationError event and no launcher Job is created
  2. with this patch, the launcher Job, Service, ConfigMap and Secret are created, and there are no worker Pods, which is the intended path

tests:

  • validation coverage for zero workers in launcher-as-worker mode
  • controller reconcile regression test
  • integration regression test

@google-oss-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign terrytangyuan for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tenzen-y

Copy link
Copy Markdown
Member

@immanuwell Could you sign the commit for the DCO check?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants