Question about Partitions And PartitionMapping • For Assets, how can we map many Upstream Partitions to Single Downstream Partition such that Upstream Asset may materialize X number of times, but downstream only materializes Once. ◦ I have seen this happening for scenario {Upstream Asset with DailyPartition} --> {Downstream Asset with Weekly Partition}, the Downstream asset executes/materializes only once for 7 materializations of Upstream Daily Partitions I am asking because I want to adopt similar flow for Upstream Assets with Dynamic Partitions and Downstream Asset just Singular Partition

Hi Binoy, Do you mean “downstream only materializes once [by the Automaterialization Daemon]“?

So that part I am not sure, if it is Automaterialization Daemon, This was an observation for DailyPartition --> WeeklyPartition Assets materialization and logs statement that I put to track how many times DailyPartition’ed Asset was called and How many times WeeklyPartition’ed Asset was called and it appeared to be 7:1 and I wanted to know how it is done, since I wanted to apply same concept to a different partitioning scheme

This was an observation for DailyPartition --> WeeklyPartition Assets materialization and logs statement that I put to track how many times DailyPartition’ed Asset was called and How many times WeeklyPartition’ed Asset was called

Something has to be launching materializations, whether it be a person doing it manually, a schedule, a sensor, or the automaterialize daemon. Can you check the “Deployment” of the UI and see if “Auto-materializing” is enabled?

I run it as

dagster dev

I’m a little unclear on what the problem is-- there are different ways to trigger materializations. You could run your weekly partitioned asset on a weekly schedule and the daily one on a daily schedule, and IIUC that would do what you’re asking for.

I think I understood Partitions a bit differently, I was falsely under impression that Partitions are causing to converge from multi to single executions between upstream and downstream assets, but in fact it’s my misguided thought process. What is really happening is: For 2 Assets

Asset_D

and

Asset_W

[ `Asset_D`:DailyPartition and `Asset_W`:WeeklyPartitions each has its own Job definition at Daily and Weekly partition scheme and both are running at their respective frequency. Convergence is just a perception 🤷‍♂️

Ah yes, yeah you are free to schedule the materialization of differently partitioned assets however you want, when they are executed is a separate concern from the mapping between them.