Hello all! Apologies in advanced if this is stated...
# ask-community
Hello all! Apologies in advanced if this is stated somewhere in the document (I couldn't find it). But I need help to schedule a job that has dependency on another job. For example, I have Job A that runs at 12AM daily, and Job B will run at 2AM daily, however, Job B should only run if Job A completed successfully as there are data which it depends from Job A. Could someone let me know how I could configure it in Dagster? Thanks in advance! cc @Kyle Downey
@sandy this the same project we were talking about re: SDA about a month ago. We have still not migrated, so if that's part of the puzzle it would be good to know.
One option would be a schedule with a custom
You would have to use the graphql api to check on the status of the partition (or just the last run) for the other job, and yield a
if it failed
What about a sensor?
The 2AM requirement leans me towards schedule since it’s primarily time based. If that isn’t crucial, then yes a sensor sounds right
Thank you for the responses! And yes it is time based + dependent on another job's status due to data dependency. @johann Will take a look at how to implement with your suggestion
If the upstream job completes at 2:47 AM (or any arbitrary time after 2 AM), would you want the downstream job to run? If so, then I think a sensor would be the right approach. Schedules make sense when you want a job to run at a particular time, but if the timing of the job is flexible, then a sensor makes the most sense