Hi team, I read that <default-run-launcher> spawns...
# announcements
Hi team, I read that default-run-launcher spawns a new process on pipeline’s repository nodes. Does it mean the “Run worker” in this case will also be a process on the node instead of a K8s job please?
Hi Hebo, that's right - you can use the K8sRunLauncher or the CeleryK8sLauncher if you'd like the run worker to be a K8s job
The Kubernetes docs have some examples where the run worker is a K8s job here: https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm
Hi Daniel! Thanks so much for the reply! I am actually trying to avoid spinning up new pods as our infra currently is solely Terraform based and doesn’t support dynamically creation of jobs/pods.
I am planning to use default-run-launcher first and then maybe explore a way to use a custom launcher to use a fixed pool of pods.