Drew You
03/31/2023, 4:13 PMMultiPartitionMapping
that is more-or-less just the identity mapping.
@asset(
required_resource_keys={"duckdb"},
ins={
"historical_daily": AssetIn(
partition_mapping=MultiPartitionMapping(
{
"days": DimensionPartitionMapping(
dimension_name="days",
partition_mapping=TimeWindowPartitionMapping(
start_offset=-window, end_offset=0
)
),
"dynamic_partition": DimensionPartitionMapping(
dimension_name="dynamic_partition",
partition_mapping=IdentityPartitionMapping()
),
}
)
)
},
partitions_def=MultiPartitionsDefinition(
{
"days": DailyPartitionsDefinition(
timezone="Etc/UTC",
start_date=spec[0]["start_date"]["inner"],
end_offset=-1
),
"dynamic_partition": dynamic_partition
}
))
With the upstream asset having more-or-less the identical partitions_def as the downstream. I'm getting the following error.
Selected assets must have the same partitions definitions, but the selected assets have different partitions definitions:
Multi-partitioned, with dimensions:
Days: Daily, starting 2019-09-10 Etc/UTC. End offsetted by -1 partitions.
Dynamic_partition: Dynamic partitions: "dynamic_partition": {AssetKey(['key'])}
Multi-partitioned, with dimensions:
Days: Daily, starting 2019-09-10 Etc/UTC. End offsetted by -1 partitions.
Dynamic_partition: Dynamic partitions: "dynamic_partition": {AssetKey(['other_key'])}
claire
03/31/2023, 10:53 PMdefine_assets_job
are not equivalent python objects. Would you mind sharing partitions def of the upstream asset so I can reproduce this?
I think if you did
assert key.partitions_def == other_key.partitions_def
this assertion would fail, I'm curious to see whyDrew You
03/31/2023, 11:42 PM