https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jay

08/18/2023, 7:03 PM
Hello Team, I hope you are doing well. I am currently working with some auto-materializing assets and am running into an issue where the upstream asset, say A, has materialized but the auto-materialize, B, (which depends on A) has not materialized. A and B have the same partitioning schema (Multi-partition of company|date). I was testing this locally with another asset C, and it works but only with the latest partition. The only difference between A and C is that C has no upstream dependencies whereas A does. Any idea on why this would be happening? I am currently on dagster 1.3.10 and B has non_argument_dep(A). Thanks in advance
🌈 1
o

owen

08/18/2023, 10:38 PM
Hi @Jay! There have been several changes to the auto-materialize system since 1.3.10, and in particular in that version and prior, only the most recent time partition of any time-partitioned asset would be materialized. I would definitely recommend updating to the latest version if possible.
p

Pablo Beltran

08/26/2023, 3:58 AM
Hi @owen I was trying to see when this behavior changed to plan the update but I don't see any mention of it anywhere and the docs still claim that this is true: https://docs.dagster.io/concepts/assets/asset-auto-execution#auto-materialization-and-partitions
o

owen

09/07/2023, 10:13 PM
Hi @Pablo Beltran -- looks like that's an oversight in the docs, but the current default behavior is that, if there are multiple time partitions that might be updated, to only update the most recent one of those partitions. (i.e. if 01-02, 01-05, and 01-07 all have had their parents updated since last tick, only 01-07 will be requested)
p

Pablo Beltran

09/07/2023, 10:14 PM
Is there any flexibility in the behavior currently?
o

owen

09/07/2023, 10:15 PM
you are able to set the
AutoMaterializePolicy.eager(max_materializations_per_minute=...)
parameter to a higher number to allow more partitions to be kicked off per iteration of the daemon
👍 1
p

Pablo Beltran

09/07/2023, 10:16 PM
Would I be able to kick of the first date first instead of the most recent date?
o

owen

09/07/2023, 10:17 PM
that's not currently possible
👍 1
p

Pablo Beltran

09/07/2023, 10:17 PM
Context is for assets that depend on the previous date it would be nice to be able to do it in order
o

owen

09/07/2023, 10:17 PM
ah well actually the sort order does go backwards for self-dependent assets
🌈 1
p

Pablo Beltran

09/07/2023, 10:17 PM
Sweet thanks!
blob salute 1