https://dagster.io/ logo
Powered by
#dagster-support
Title
# dagster-support
a

Andras Somi

01/22/2023, 2:19 PM
Hi, I’m trying to write a custom 
io_manager
that saves the output of a partitioned asset and behaves nicely if a non-partitioned downstream asset asks for the output (it should return the latest available partition), but I got a bit lost on what to look for in the 
InputContext
object. I tried to decipher the relevant methods on 
UPathIOManager
but couldn’t really grasp what’s going on with the partition keys (comments are confusing for me, I even sense some typos there, but cannot be sure…). How should I decide whether the downstream asset is partitioned so 
load_input
should return the specific partition or non-partitioned therefore 
load_input
should return the latest available?
Hm, probably I chased myself down to a rabbit hole with 
UPathIOManager
. I ended up having something like this in 
load_input
:
Copy code
if context.has_partition_key:
    # Asking for a specific partition
    date_str = context.partition_key
    # ... retrieve output for this key
else:
    # Asking for latest partition
    # ... check all available partitions and retrieve latest
Is this how it should be done?
y

yuhan

01/28/2023, 1:49 AM
Hey Andras, sorry for the delay. Yes I’d recommend the same. Having the condition check in load_input looks reasonable.
a

Andras Somi

01/28/2023, 7:19 AM
Great, thanks for confirming!
Powered by