https://dagster.io/ logo
#ask-community
Title
# ask-community
r

Robin Sinha

07/19/2023, 6:39 AM
Hi Team, We have a dynamic job (lets say A) which executes the same op in parallel, and another job (lets say B) which starts by a run status sensor (SUCCESS) of job A. Sometimes, some of the ops in parallel failed in job A due to
ChildProcessCrashException
. Although, rest of the ops in job A executed successfully job B will not start since it is marked as FAILURE but I want it to run anyway. One approach to achieve this is to add another run status sensor (FAILURE) of job A but I don't want to do that since there may be some overhead of additional sensor. Is there anyway to achieve it more efficiently with or without a sensor?
a

alex

07/21/2023, 3:05 PM
failure sensor should be fine - what type of overhead are you concerned about? what have you measured?
r

Robin Sinha

07/24/2023, 4:31 AM
I haven't measured anything yet. But, I am concerned about memory, and compute usages of an additional sensor since sensor checks job status every 30s and these jobs are sometime finishes quickly and sometime take significant amount of time. Is there any sensor which works regardless job is failed or success?
a

alex

07/24/2023, 4:03 PM
currently
run_status_sensor
only allows tracking one type of run status.
2 Views