Hey Dagster community: I have an op that schedules...
# ask-community
d
Hey Dagster community: I have an op that schedules a k8s job using
execute_k8s_job
. The op is called via
map
of a `DynamicOutput`list.
execute_k8s_job
uses the name from the
OpExecutionContext
as the k8s job name. However, it doesn’t appear that the op’s name is actually unique for each invocation of the op in the map operation, which results in dagster attempting to create k8s jobs with duplicate names. I’ve considered using a factory that generates ops with unique names, but am unsure how I’d use this with
map
. How do I ensure that each mapped invocation of an op has a unique name?
o
Hi @Daniel Chalef! This is an issue on our end -- I just threw together a PR on our end to ensure that these job names are unique when the
execute_k8s_job
op is downstream of a dynamic output. This fix should make it out in this week's 1.1.19 release
d
Wow. That’s awesomely quick! Thank you. Do you have a recommended workaround in advance of the release? I was hoping to get my relatively simple pipeline running today 🫤
o
hm I don't think there's any particularly nice workaround in the short term, other than just essentially copy-pasting the new implementation (it's a 1-line difference: https://github.com/dagster-io/dagster/pull/12344/files) and using that until the release comes out
d
Thanks. Will consider dropping this into my local install.
Works perfectly. Thanks again!
o
blob salute