Hi - I've noticed that when running a job with the

k8s_job_executor

(i.e. each op is run as its own job), cancellation from the UI manages to delete the "parent" k8s job, but its children (the op jobs) silently continue in the background.I'm not sure that this was always the case, but I think you can tighten this up by including an owner reference (https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/#owner-references-in-object-specifications) when creating the op jobs here: https://github.com/dagster-io/dagster/blob/eb26ca5d1fb69df0034742630e8fcc3f6799c34b/python_modules/libraries/dagster-k8s/dagster_k8s/executor.py#L234-L255 So when a user cancels from the UI, the

K8sRunLauncher

deletes the

dagster-run-...

job: https://github.com/dagster-io/dagster/blob/eb26ca5d1fb69df0034742630e8fcc3f6799c34b/python_modules/libraries/dagster-k8s/dagster_k8s/launcher.py#L313-L315, and then kubernetes itself will delete the 'owned'

dagster-step-...

jobs automatically. Loving all the work you guys are doing!

Thanks for the appreciation, Agon! Do you mind creating a GH issue to track this issue? It’ll help us with some planning and also act as a signpost for other community members running into this.

Thanks @prha - have tried to capture this in: https://github.com/dagster-io/dagster/issues/14248

🙏 Thank you!