Hi All, I have a Dagster job that executes certain Ops in parallel. These Ops have varying run-times and for my use case it makes sense to fail the Job immediately if any one of the parallel Ops fail. The current behaviour is that Dagster waits for all parallel Ops to finish before failing the Job. How might I achieve this?

Hi Sanidhya. Here's a suggestion from a similar question in the past:

One thought that came to mind (although I haven't tried it myself) is to create a failure_hook that fires off something like

context.instance.run_coordinator.cancel_run(context.run_id)

. Then you can apply this hook to every step on your job using the hooks parameter on the job decorator (i.e.

@job(hooks={my_cancel_hook})

)

10:00 AM

there might be some subtitles there where this could end up sending a bunch of cancelation requests at once, so you might need to add some extra checks around that.