https://dagster.io/ logo
Title
s

scott simpson

12/14/2022, 9:19 AM
Hi All. I'm using a couple of custom PartitionMappings to map between some different Multipartitions. It works great! Since 1.1.6, I've gotten the warning
DeprecationWarning: Non-built-in PartitionMappings, such as MarketToDatePartitionMap are deprecated and will not work with asset reconciliation. The built-in...
Am I still going to able to have custom PartitionMappings in the future? Or is this feature just going away for asset reconciliation sensors?
1
j

jamie

12/14/2022, 8:11 PM
cc @sandy
s

sandy

12/14/2022, 10:32 PM
Hi Scott - we don't have any near-term plans to eliminate it. How do your custom PartitionMappings map between multi-partitions? Asking because we're interested in building in PartitionMappings that do this.
s

scott simpson

12/16/2022, 7:32 AM
Thanks for confirming. I have a hierarchical partition structure, a subset is below: Some assets need to materialise for date|chain, others for date|market, some just for date I'm using the PartitionMappings to make the dependencies work between the assets with different partition schemes
p

Peter Davidson

01/20/2023, 3:12 PM
@scott simpson any chance you could share a code snippet? Trying to do exactly the same thing
s

scott simpson

02/11/2023, 1:26 AM
Sorry @Peter Davidson, missed this. I ended up skipping the partition maps and lived with a mild duplication of data. Non-standard partition maps aren't supported with freshness policies (and hence the whole declarative scheduling thing) so I standardised on a single partition definition.
:rainbow-daggy: 1
p

Peter Davidson

02/13/2023, 6:10 AM
this is where i landed too. some duplication of data but not a disaster
👌 1