Casper Weiss Bang
08/17/2023, 1:30 PMSkip condition if [...] a run is already running
or something. It isn't there now (Version 1.4.3).
We have a job that runs every morning, i.e A -> B -> C
. B has an automaterialization policy. Well when the job finished A, but before B started, the policy kicked in, started to materialize asset B. well when the pipeline then continued we had a concurrency issue, the job (due to B) failed, which meant that C never got materialized.
I assume this is a bug? shouldn't the auto materialization policy back off if the asset is already in an active run? It's a non partitioned asset (B)owen
08/17/2023, 10:26 PMCasper Weiss Bang
08/21/2023, 5:05 AMowen
08/25/2023, 9:01 PMowen
08/25/2023, 9:01 PMowen
08/25/2023, 9:10 PMCasper Weiss Bang
08/28/2023, 5:12 AMit has to do with the fact that your job is manually launchedMight be because we are not on the latest version, and the auto materialization assets are registered as manual 🙂 i'll update. Thanks for tracking it though! awesome work