Hey, I am having an issue with partition mapping. I am using the AllPartitionsMapping and is working as expected. But only if all partitions of the upstream asset succesfully are materialized. As soon as a single partition is "defined" but not exisiting in storage, the asset will fail with a dagster._core.errors.DagsterExecutionLoadInputError. I would like the downstream asset to simply take all the "existing" partitions as input and ignore the failed ones. I am using dynamic partitions. Is there a way to do it by using dagster classes or should I write my own class which does the job. I am hesitating do so, because it says, that the abstract class "PartitionMapping" can change at any time .

hi @Frederik Löw! I think the proper place to handle this logic would be within your IOManager. Many built-in IOManagers assume that all upstream partitions will be available, hence this error, but this is not intrinsic to the PartitionMapping. If you're using the default UPathIOManager (or any IOManager that inherits from there), you should be able to avoid this error by setting:

@asset(ins={"my_input": AssetIn(metadata={"allow_missing_partitions": True})})
def some_asset(my_input):
    ...

If you're using a different IOManager, then it should still be possible to modify that base implementation with some custom logic

Hey @owen, thank you for quick help. I just tested it and it works. I Didn't even think of looking in that direction. :)