Hi all, I see a number of older issues and feature...
# ask-community
m
Hi all, I see a number of older issues and feature requests for custom calendar support in timewindow partitioning. Has there been any progress on this? I worked around this previously by using a dynamic partitions with date strings as the partition names and then building custom partition mappings to take care of things like fan in / fan out and lagging of partitions to other assets. However, in the newest versions of dagster this functionality seems to be deprecated. This is leaving me stuck as to how I can have date-like partitions with a business day schedule that can't be represented as cron-jobs. Any suggestions on ways to accomplish this?
s
hi Matt - alas, there has not been much progress on this. what exact functionality are you seeing is deprecated?
m
From 1.3.12 changelog: Breaking Changes • Legacy methods of
PartitionMapping
have been removed. Defining custom partition mappings has been unsupported since 1.1.7. Would be happy to help build something here if there are any ideas on the right way to architect it.
s
ah, got it, fwiw you can probably get the same behavior you had before by implementing the non-removed methods of
PartitionMapping
Would be happy to help build something here if there are any ideas on the right way to architect it.
I think the biggest question is the API for how the custom calendar gets specified
if you'd be up for filing a github issue with a proposal (or adding a proposal to an existing github issue), we would consider it
m
Would you be able to point me to the right documentation / docs on the new partionmapping interface?
s
it's an undocumented internal interface, so my recommendation would be to look at the code for the
PartitionMapping
I know not the most satisfying answer
m
for my purposes having it exclude certain dates would work fine. I think it may be easiest to have a superset created with cron and then an exclusion list where no asset partition is created. I don't know if there have been discussions around this. Truthfully, I'm not super familiar with the API itself and have worked mostly at higher levels with it.
s
excluding certain dates sounds like it could be a reasonable approach