Emilja Dankevičiūtė
12/21/2022, 7:31 AMop_retry_policy
on define_asset_job
? tags={"dagster/max_retries": 2}
would probably work. But I'm also interested in controlling delay between retries..Jason
12/21/2022, 2:04 PM@op(
retry_policy=RetryPolicy(
max_retries=3,
delay=0.2, # 200ms
backoff=Backoff.EXPONENTIAL,
jitter=Jitter.PLUS_MINUS,
)
)
def even_better():
fails_sometimes()
Emilja Dankevičiūtė
12/21/2022, 2:22 PMop
. We have a job that materializes assets via define_asset_job
. You can pass op retry policy (the one that you mentioned) when you define custom job via job
decorator. But not for define_asset_job
as far as I can tell.Jason
12/21/2022, 2:40 PMretry_policy
via @asset (api link included).
On the job level, I’m not sure if you can control the delay but there is a retry_strategy
tag which I don’t use on asset jobs but probably still works.
https://docs.dagster.io/_apidocs/assetsEmilja Dankevičiūtė
12/22/2022, 7:05 AMload_assets_from_airbyte_instance
& load_assets_from_dbt_project
, so no decorator. And I want to set retries based on the job and not on the asset, if possible.