https://dagster.io/ logo
#ask-ai
Title
# ask-ai
r

R Lucas

02/20/2024, 7:52 AM
Is there a PartitionMapping combining both TimeWindowPartitionMapping and LastPartitionMapping to map the last 10 hourly partition of a parent asset and materialize an unpartitioned child asset ?
s

Scout

02/20/2024, 7:52 AM
Based on the documentation, there isn't a specific
PartitionMapping
that combines both
TimeWindowPartitionMapping
and
LastPartitionMapping
to map the last 10 hourly partitions of a parent asset and materialize an unpartitioned child asset. The
TimeWindowPartitionMapping
allows you to map partitions based on a time window relative to the downstream partitions, and the
LastPartitionMapping
maps to the last partition of the upstream asset. However, the documentation does mention that you can use
TimeWindowPartitionMapping
with
start_offset
and
end_offset
to map partitions from the upstream asset to the downstream asset. For example, if you want to map the downstream partition “2022-07-04” to the upstream partitions “2022-07-03” and “2022-07-04”, you can set
start_offset=-1
and
end_offset=0
. For your specific use case, you might consider implementing a custom logic outside of the provided
PartitionMapping
options. This could involve creating a custom sensor or some other mechanism to trigger the materialization of the unpartitioned child asset based on the completion of the last 10 hourly partitions of the parent asset. If you need to materialize an unpartitioned asset based on the state of partitioned upstream assets, you may need to write custom logic to handle this, as the existing
PartitionMapping
options may not cover this specific scenario. The documentation does not provide an example of a
PartitionMapping
that directly supports this use case.
2 Views