https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Alec Koumjian

05/02/2023, 8:37 PM
I have a k8s PVC that I need mounted for some jobs. The id of this PVC depends on the environment. Currently I set this with the
dagster-k8s/config
tag in the
@job
definition. In an ideal world, I could query kubernetes for the list of matching PVC's and update the job definition at runtime so it mounts the right PVC for the environment (or if it has been updating dynamically by some other system). Is there a way to have this set dynamically at launch time?
d

daniel

05/03/2023, 12:26 AM
I don’t think there’s a way to set this dynamically other than by writing a custom run launcher that subclasses K8sRunLauncher which is not trivial If this job is being launched from a schedule or sensor, those could set the dagster-k8s/config tag value in the RunRequest that’s returned from the schedule or sensor function
a

Alec Koumjian

05/04/2023, 2:19 PM
It looks like I would also be able to do this through the graphql interface if I had a separate service initiating the run via
launchRunMutation
by setting
launchRun.executionParams.executionMetadata.tags
, does that seem right?
d

daniel

05/04/2023, 2:19 PM
Yeah that would also work!
a

Alec Koumjian

05/04/2023, 2:20 PM
Great, thank you for all of your (continued) help!
condagster 1
y

Yuval Grinberg

07/02/2023, 7:45 AM
Hello! Got stuck in a similar situation. When i add the k8s tag in the @job everything runs alright. When I override it in the sensor i get this error:
Copy code
dagster._check.CheckError: Value in Mapping mismatches expected type for key dagster-k8s/config. Expected value of type <class 'str'>. Got value {'container_config': {'resources': {'limits': {'cpu': '42', 'memory': '243G'}, 'requests': {'cpu': '42', 'memory': '243G'}}}} of type <class 'dict'>.
apparently the type check of tags is different for when you set it through the @job or via a RunRequest. Any ideas on how to overcome this?