Hey folks, in Airflow there’s a `depends_on_past` ...
# integration-airflow
d
Hey folks, in Airflow there’s a
depends_on_past
option in DAGs where a scheduled DAG run won’t kick off until the DAG run preceding it finishes successfully. What’s the recommended way to achieve this in Dagster?
j
There isn't a direct analog but depending on what you're trying to accomplish you could 1. define a
should_execute
function on your schedule 2. define a RunStatusSensor that runs whenever the last job completes Also worth noting that Dagster doesn't have as many of the same noisy neighbor issues that exist/existed in airflow(that were addressed with pooling and other features like
depends_on_past
) so if your mostly looking to avoid having a bunch of inflight jobs you shouldn't need to worry about that as much
d
Thanks! I’ll look into that
j
np! Just in case you are looking to limit the number of inflight runs of a given job there are also some ways you configuring that as well https://docs.dagster.io/deployment/run-coordinator#limiting-run-concurrency, these options work similarly to
max_active_runs
run airflow