Is there a good page for reference when writing an IO manager? I have written a few that basically take assets and put them into Postgres, and some of the challenges I've run into are: • What is the best way to pass metadata from an

op

to the

io_manager

? What if it needs to be dynamic, e.g. if the op knows what the name of the output should be (setting metadata on the

Output

does not seem to work) • What is the best way to have an

io_manager

that only really works for assets? (thanks to @owen for the idea of the custom decorator) • What are all of the possible sources of the "name" of the thing: ◦

context.asset_key

? ◦

context.step_key

? ◦ various

names

? • detecting partition type (currently I use

instanceof(context.asset_partitions_def

a bunch, maybe there is a better way) • better understanding how

context.asset_partitions_time_window

• How does all of this work in the context of: ◦ Backfills ◦

@multiasset

◦

SourceAsset

◦

DynamicOutput

• Other things to be aware of that may contain sharp edges for those who are writing

io_manager

s Currently what I do is look at the

dagster

test suite, which is pretty awesome, but sometimes doesn't give the comprehensive overview that I sometimes lack. Does something like this already exist/can it?

hey! We don't really have any content like this right now - but it is on the radar / planned at some point. cc @jamie who might have more details, or at the very least might find the challenges presented here to be good material

hey @Daniel Mosesson - i dont have much to add at this point. writing a guide like this is on my to do list as part of some other IO manager changes I have in the works. I’ve copied over all of the points you brought up so i can address them in the future guide. If you do have any specific questions as you write io managers, i’m happy to help out or point you to examples in our own IO managers