https://dagster.io/ logo
#ask-community
Title
# ask-community
z

Zeeshan Nehal

07/28/2023, 1:45 AM
Hello everyone, I need your expert comments to create a pipeline in Dagster using the asset. The expected partition should be based on 'id' and 'date,' where each 'id' will have specific dates for partitioning. Currently, I am able to use
MultiPartitionDefinition
for
DynamicPartitionDefinition
to pull 'id's from the database at runtime. However, using
TimeWindowPartitionDefinition
for handling dates is not helpful, as it sets universal partition dates, causing issues with backfill and not meeting our requirements. I have attached a drawing for your consideration. I tried to contact the Dagster team, but unfortunately, they were not very helpful.
s

sandy

07/28/2023, 11:26 PM
Hi @Zeeshan Nehal - having different dates ranges for each element on another axis in a multi-partitions definition is not currently supported.
z

Zeeshan Nehal

07/29/2023, 7:01 AM
@sandy thank you for reply. Any other solution can you recommend to achieve our desired results?
2 Views