Hey everyone, is there a way to define the default...
# deployment-kubernetes
f
Hey everyone, is there a way to define the default k8s resources on all asset materialization jobs without having to define a job from each asset and manually applying the k8s tags there?
d
Hey fahad - we have a way to set this for all jobs, but I don't think we have a way to specifically differentiate asset materialization jobs from other jobs, would that still help here?
f
Being able to set a global default for all jobs would be good enough for my use-case!
(or rather per Code Location/Definition)
d
For per-code-location I think the resources key here would affect every pod spun up for that code location https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm#step-61-configure-the-deployment
f
I remember trying this and it seemed to only apply to the user-code deployment pod, but not launched dagster-run pods. Let me give this a quick try though
d
What version of dagster are you on currently?
The "includeConfigInLaunchedRuns" flag mentioned there is relatively new
ideally you'd be able to separately list the resources for the server and any launched runs, but that isn't quite there yet
❤️ 1
f
ooo ok I missed that flag. Looks like the user-code deployment is on the latest release of dagster
d
I think if your helm chart is past 0.15.0 or so then that behavior should be enabled by default
f
The daemon and dagit are deployed separately from our user-code deployments so need to confirm versions there - not sure if that would have an impact
ok that worked for the time being!
ideally you’d be able to separately list the resources for the server and any launched runs, but that isn’t quite there yet
Would definitely be really neat to be able to specify this within the
Definitions
or something! But thanks for the help!!
d
you mean just for the runs? There's a bit of a chicken/egg problem if it applies to the code servers since you'd need to start the code server first
f
Exactly, just for the runs so that the user code server doesn’t have to sit at the resources meant for dagster-run pods
But it would be nice to specify defaults for dagster-run/dagster-step pods somewhere that also encompasses asset materializations (i.e. not just
Job.tags
) like you said