https://dagster.io/ logo
#ask-ai
Title
# ask-ai
s

Shane Keller

02/27/2024, 11:20 PM
Can dagster guarantee that an op without a retry policy is executed exactly once? Or is it executed at least once?
s

Scout

02/27/2024, 11:20 PM
In Dagster, if an op does not have a retry policy specified, the default behavior is to execute it exactly once during a pipeline run. However, it's important to note that this does not guarantee that the op will be executed only once in all circumstances. For example, if the entire job is retried due to a failure in another op or if the job is manually re-executed, the op without a retry policy may be executed again as part of those new runs. The documentation provides mechanisms for retrying ops that fail, such as using a
RetryPolicy
or raising a
RetryRequested
exception within the op. These mechanisms are used to explicitly define the retry behavior when an op encounters transient issues. Without such a policy, the op is not retried upon failure within the same run, and it is expected to execute exactly once. However, it's important to distinguish between op retries and run retries. Run retries can be configured to retry the entire job or specific steps if a run fails, which could lead to an op being executed more than once across different runs, even if it does not have a retry policy. In summary, without a retry policy, an op is intended to execute exactly once within a single run, but there are scenarios, such as job re-executions or run retries, where the op could be executed more than once.
2 Views