AutoMaterializePolicy

creates a Run for 5 assets from which 2 are currently being materialized - is this normal? It shouldn't be, right?

hi @Daniel Gafni! can you share a bit more about the shape of the graph you're working with / which assets were getting updated and which were current materializing?

Hi, We have experienced the same behavior with our setup. Asset is being materialized and while it still running a new run is created.

A -> B B -> (C, D) (B, C) -> E A gets materialized by a sensor (C, D) is a

@graph_multi_asset

What's the timeline here? Is it something like: 1. A is materialized 2. run is kicked off for B, C, D, E 3. a duplicate run is kicked off for B, C, D, E? Or are there other events happening in the middle here? One thing to note is that there is (currently) no special handling for cases where an asset is in progress, all else being equal. I.e. if you have a simple graph

A -> B

, and A takes 1 second to complete, while B takes 30 minutes, and B has an eager policy, then materializing A the first time will of course kick off a run of B, but each materialization of A before that first run of B completes will also kick off a run of B. It sounds like that might not be the semantics you're expecting here, is that fair?

@owen, I observe following behavior: 1. Asset graph is sequential: A -> B -> C -> D -> E 2. E has

Freshness policy

defined and Lazy

AutoMaterializePolicy

, rest of assets don't have policies 3. Assets A, C, D, E are fast, B is long running 4. Run is triggered for assets A, B, C, D, E based on Freshness policy 5. A is materialized 6. While B is being materialized, another run is triggered for C and D only, no extra runs for E It looks like

AutoMaterializePolicy

is not considering planned materializations

@Nikolaj Galak I'm guessing in this case, the other assets also have AutoMaterializePolicies, they just don't have freshness policies (otherwise the daemon would never kick off a run for A B C and D), is that accurate? Also, are there any source assets in your setup? The policy is definitely intended to take planned materializations into account, but a bugfix is going out in this week's release that patches up a case where that wasn't happening properly (involving assets that had both source asset parents and non source asset parents)

@owen, yes, you are right, lazy

AutoMaterializePolicy

is set for all the assets, not just E.

A

is source asset and

B, C, D, E

are dbt assets in my example. Great to hear that bugfix is on its way, appreciated!

yep -- hard to tell if this is exactly the same case, but definitely let me know if things seem resolved after the latest release, and if not we can dig in more!

Hi @owen, we are still observing the behavior where another run of asset with AutomaterializePolicy on. First run finished about 5 seconds after second run was started. We are running version 1.4.10