Hi all, is there a similar inbuild dagster function much like DailyPartitionsDefinition that however takes in a start and end date and partitions in that range?

I don’t believe

TimeWindowPartitionsDefinition

and its derivatives take in an end date - they currently return an infinite generator given the cron schedule and the start date. @sandy are there any work arounds for this? In the meantime, feel free to file a feature request for this

The final goal I am drying to achieve is: If I partition on start day "x", I would like it to retrieve also the data from the day before. Something like a railing window partition.

You can achieve something like that with partition mappings. Here's an example someone posted a little while ago: https://dagster.slack.com/archives/C01U954MEER/p1658350429504999?thread_ts=1658345537.413359&cid=C01U954MEER. Would this work for you?

Hi Sandy, Extension to this, are there any challenges which would arise if an asset was dependent on a past materialisation of itself? For example, to calculate something like a cumulative sum, the options would be: • Load all prior materialisations of the upstream asset • Load the new data in the latest partition of the upstream asset, plus the cumulative sum as of the previous materialisation of the cumsum asset (NDaysPartitionMapping(days = 1, offset = 1) of self)) Wondering whether or not this would violate the acyclic part of a DAG...in theory it is only ever reliant on a prior partition, so it isn't explicitly acyclic but could imagine this might lead to some issues!

Thank you @sandy this was really helpful.

hey @Matt - I missed this. I don't think an asset depending on a past materialization of itself currently works, but I'd like to support that in the future

Ah, fair enough. Is there some workaround we could do in the meantime? We're currently just dumping to an external summary table but it means we end up with the potential for lots of fun weird states or race conditions. Fine while we are building out but probably not the most robust solution!