My colleague has a problem with

AllPartitionMapping

during upstream backfills. He doesn't want to perform a full backfill, instead he wants to work with let's say 1 week of partitions from 2 months ago. Of course, the

AllPartitionMapping

doesn't like the fact that not all upstream partitions are materialized. Meanwhile my colleague wants it to act like "load all partitions that I materialized previously". I think it should be possible to do with a different partition mapping which would query Dagster for partitions that have ever tried to be materialized (both successfully and not) and then get the latest partition key from this list. Any tips on how to implement this? Edit: I realized the downstream asset should probably just be partitioned too

currently, we expect PartitionMappings to execute without looking at what's already materialized would it be possible for your IO manager to avoid reading partitions in the case that they don't exist?

It’s possible (and UPathIOManager supports it), but then you can’t distinguish non-yet-materialized partitions from partitions that failed to materialize. Anyway, we already solved it by making the downstream asset partitioned too.