Hi i have a questions, i am using dagster k8s, and...
# deployment-kubernetes
Hi i have a questions, i am using dagster k8s, and i have 2 workspace (2 images), and i would like that each will spin a job by its image with different VM type (lets say in GCP). is it possible in dagster? is it can be found in the helm files (at your github repository)? i will appreciate for any help
i would like to add more information: • we would like to have an option to spin a specific VM for specific pod that will be exist on just run time and then will be down until next run • i wonder if it can be done by node selector
Hi igor - is this something that can be configured in the job/pod that's spun up by dagster? If so the configuration options here may help that let you customize the job and pod that is spun up by the K8sRunLauncher: https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#job-or-op-kubernetes-configuration If those aren't sufficient this may be a case where you would write your own RunLauncher, which is a pluggable part of the system
10x i will try it
The node selector is what you want, with the label
set to the name of the node pool you want to use
As long as you have autoscaling enabled, cluster-autoscaler will take care of it from there.
(k8s itself doesn't reeeeeeally know about "node type" directly)