[backend] conditions in component.yaml does not work properly #11009
Description
Environment
- How did you deploy Kubeflow Pipelines (KFP)?
Kubeflow Platform (EKS) - KFP version:
v2.2.0 - KFP SDK version:
v2.8.0
Steps to reproduce
Conditions in the component.yaml file are not working correctly. It appears that the argo-workflow component takes the "IF" condition argument literally. This is something that worked in kubeflow v1.8.0 and KFP 1.8.X (argoworkflow manifest)
here we I wrote dummy pipeline that prints /etc/hosts file. If line_number=true it should add -n argument to cat executable to print file with line numbers.
- Compile following using KFP v2.8.0
---
name: kfp-busybox
description: check-conditions
inputs:
- name: file
type: String
default: "/etc/hosts"
- name: line_number
type: Boolean
optional: true
implementation:
container:
args:
- inputValue: file
- if:
cond:
isPresent: line_number
then:
- -n
image: busybox:latest
command: [
"/bin/cat"
]- compiled component:
components:
comp-kfp-busybox:
executorLabel: exec-kfp-busybox
inputDefinitions:
parameters:
file:
defaultValue: /etc/hosts
isOptional: true
parameterType: STRING
line_number:
isOptional: true
parameterType: BOOLEAN
deploymentSpec:
executors:
exec-kfp-busybox:
container:
args:
- '{{$.inputs.parameters[''file'']}}'
- '{"IfPresent": {"InputName": "line_number", "Then": ["-n"]}}'
command:
- /bin/cat
image: busybox:latest
pipelineInfo:
name: pipeline
root:
dag:
tasks:
kfp-busybox:
cachingOptions:
enableCache: true
componentRef:
name: comp-kfp-busybox
inputs:
parameters:
file:
componentInputParameter: file
line_number:
componentInputParameter: line_number
taskInfo:
name: cat-hosts
inputDefinitions:
parameters:
file:
parameterType: STRING
line_number:
parameterType: BOOLEAN
schemaVersion: 2.1.0
sdkVersion: kfp-2.8.0
---
platforms:
kubernetes:
deploymentSpec:
executors:
exec-kfp-busybox:
imagePullSecret:
- secretName: sdk-docker- upload pipeline to kubeflow platform
- run pipeline
Output:
time="2024-07-16T11:16:04.035Z" level=info msg="capturing logs" argo=true
time="2024-07-16T11:16:04.153Z" level=info msg="capturing logs" argo=true
I0716 11:16:04.212214 25 launcher_v2.go:90] input ComponentSpec:{
"inputDefinitions": {
"parameters": {
"file": {
"parameterType": "STRING",
"defaultValue": "/etc/hosts",
"isOptional": true
},
"line_number": {
"parameterType": "BOOLEAN",
"isOptional": true
}
}
},
"executorLabel": "exec-kfp-busybox"
}
I0716 11:16:04.213720 25 cache.go:139] Cannot detect ml-pipeline in the same namespace, default to ml-pipeline.kubeflow:8887 as KFP endpoint.
I0716 11:16:04.213766 25 cache.go:116] Connecting to cache endpoint ml-pipeline.kubeflow:8887
# Kubernetes-managed hosts file.
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
cat: can't open '{"IfPresent": {"InputName": "line_number", "Then": ["-n"]}}': No such file or directory
I0716 11:16:04.443096 25 launcher_v2.go:151] publish success.
F0716 11:16:04.443142 25 main.go:49] failed to execute component: exit status 1
time="2024-07-16T11:16:05.156Z" level=info msg="sub-process exited" argo=true error="<nil>"
Error: exit status 1
time="2024-07-16T11:16:06.037Z" level=info msg="sub-process exited" argo=true error="<nil>"
Error: exit status 1
as you can see here:
cat: can't open '{"IfPresent": {"InputName": "line_number", "Then": ["-n"]}}': No such file or directory
Expected result
Print file with line numbers for instance:
1 127.0.0.1 localhost
2 ::1 localhost ip6-localhost ip6-loopback
3 fe00::0 ip6-localnet
4 ff00::0 ip6-mcastprefix
5 ff02::1 ip6-allnodes
6 ff02::2 ip6-allrouters
Materials and Reference
- https://www.kubeflow.org/docs/components/pipelines/reference/component-spec/
- test example that should cover this usecase: https://github.com/kubeflow/pipelines/blob/master/backend/src/apiserver/server/test/xgboost_sample_pipeline.yaml#L304
Impacted by this bug? Give it a 👍.