https://dagster.io/ logo
Join Slack
Powered by
is there a way to yield multiple DynamicOutputs fr...
# ask-community
z
is there a way to yield multiple DynamicOutputs from an op? I haven't been able to find anything in the docs, and I'm stumbling when trying to figure out what I would use for the 
output_defs
argument of the op generating the DynamicOutputs - this section is a bit ambiguous but maybe seems related, although I'm suspecting the 
multiple_dynamic_values
op used in the example is supposed to be an op that operates on DynamicOutputs, not one that yields them
🤖 1
j
hi @Zach i think this should be possible. can you send me any code snippets of what you've tried? I can see about putting together a quick example too
z
sure -
Copy code
@op(
    name="args_generator",
    config_schema={"chroms": Field([str])},
    out={'out1': DynamicOut(str),
         'out2': DynamicOut(str)},
)
def generate_args(context: OpExecutionContext):
    for c in context.op_config["chroms"]:
        yield DynamicOutput(c, mapping_key=c), DynamicOutput(c, mapping_key=f'{c}_1')
this gives a 
AttributeError: '_args_generator_outputs' object has no attribute 'map'
on compilation
j
can you also share where 
generate_args
is being called in a graph/job?
z
Copy code
@graph()
def example_io_graph():
    generate_args().map(do_processing)
j
ok great! working on a repro now and i'll let you know what i find
z
ah I see that I can do
Copy code
@graph(name="example_io_dnax")
def example_io_graph():
    arg1, arg2 = generate_args()
    arg1.map(do_processing)
    arg2.map(do_processing)
so maybe if I need multiple values going to 
do_processing
they should just be passed as a dictionary or tuple... I guess a refinement of the question would be whether arg1 and arg2 in the example above can be fed to 
do_processing
together, or if by returning two DynamicOutputs you basically are creating two separate downstream branches in the dag
j
yeah i was about to send you a similar snippet! conceptually, i think of two DynamicOuts as creating separate branches in the dag, like you mentioned. we dont have a way to zip two lists of DynamicOuts together so they can be passed in pairs to downstream ops since you can just yield out tuples in your DynamicOuts instead.
z
okay cool, that makes sense and seems reasonable after walking through it. thanks for your help!!
6 Views
Open in Slack
PreviousNext
Powered by