⚒️Kubeflow Pipelines

Machine Learning Pipelines for Kubeflow

KFP(Kubeflow Pipelines)

GitHub - kubeflow/pipelines: Machine Learning Pipelines for KubeflowGitHub

Kubernetes 환경에서 MLOps 구성을 위해 Kubeflow 사용 시 가장 유용한 컴포넌트 중 하나

The Kubeflow pipelines service has the following goals:

End to end orchestration: enabling and simplifying the orchestration of end to end machine learning pipelines
Easy experimentation: making it easy for you to try numerous ideas and techniques, and manage your various trials/experiments.
Easy re-use: enabling you to re-use components and pipelines to quickly cobble together end to end solutions, without having to re-build each time.

Standalone Installation

The installation process for Kubeflow Pipelines is the same for all three environments covered in this guide: kind, K3s, and K3ai.

# env/platform-agnostic-pns hasn't been publically released, so you will install it from master
export PIPELINE_VERSION=1.8.5
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"
kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=$PIPELINE_VERSION"

KFP 2

export PIPELINE_VERSION="2.0.0-alpha.4"

kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"
kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/dev?ref=$PIPELINE_VERSION"

kubectl port-forward svc/ml-pipeline-ui -n kubeflow 8888:80 --address 0.0.0.0

Uninstall Pipelines

export PIPELINE_VERSION=1.8.5
kubectl delete -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=$PIPELINE_VERSION"
kubectl delete -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"