https://dagster.io/ logo
Join Slack
Powered by
Hi i have a questions, i am using dagster k8s, and...
# deployment-kubernetes
i
Hi i have a questions, i am using dagster k8s, and i have 2 workspace (2 images), and i would like that each will spin a job by its image with different VM type (lets say in GCP). is it possible in dagster? is it can be found in the helm files (at your github repository)? i will appreciate for any help
i would like to add more information: • we would like to have an option to spin a specific VM for specific pod that will be exist on just run time and then will be down until next run • i wonder if it can be done by node selector
d
Hi igor - is this something that can be configured in the job/pod that's spun up by dagster? If so the configuration options here may help that let you customize the job and pod that is spun up by the K8sRunLauncher: https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#job-or-op-kubernetes-configuration If those aren't sufficient this may be a case where you would write your own RunLauncher, which is a pluggable part of the system
i
10x i will try it
n
The node selector is what you want, with the label 
<http://cloud.google.com/gke-nodepool|cloud.google.com/gke-nodepool>
set to the name of the node pool you want to use
As long as you have autoscaling enabled, cluster-autoscaler will take care of it from there.
(k8s itself doesn't reeeeeeally know about "node type" directly)
Open in Slack
PreviousNext
Powered by