Andras Somi01/11/2023, 9:26 PM
errors as the downstream asset seems to look for the missing outputs in the incomplete partitions. I would have expected them to simply be missing from the partition dict or have
as value. I can return a “dummy” value if the upstream file is missing and check for it in the downstream asset, but that would kinda defeat the purpose of an optional asset out. Am I missing something here or is this really not doable?
sandy01/12/2023, 1:11 AM
Andras Somi01/12/2023, 9:44 AM
flag in metadata: https://docs.dagster.io/concepts/io-management/io-managers#custom-filesystem-based-io-manager Thanks for pointing me the right direction, I’m sure I couldn’t have found this.
sandy01/12/2023, 4:21 PM