https://dagster.io/ logo
Join Slack
Powered by
I have a question about configuring graphs to perf...
# ask-community
a
I have a question about configuring graphs to perform conditional branching. The idea is that I want to apply the "conditional branching" pattern described here, but where (1) the conditional branching is applied once to each element of a dynamic-output op, and (2) the results all get combined together in a downstream op. I'm attaching a basic example of this setup. The problem is that if I configure 
make_list
to generate outputs that will hit only branch A or only branch B (for example, 
{n_min: 0, n_max: 1}
or 
{n_min: 1, n_max: 2}
), the downstream op 
combine_lists
gets skipped completely because it thinks one of its inputs was skipped. I was hoping in this case that 
mapped_a.collect()
and 
mapped_b.collect()
would behave like empty lists when used as inputs to a downstream op, instead of skipping the downstream op altogether. Is this a bug? If this is the intended behavior, is there any way to accomplish what I'm aiming for?
example_job.py
🤖 1
Update: I tried adding a "default_value" to each of the inputs for the "combine_lists" op, but that unfortunately didn't change the behavior:
Copy code
@op(
    ins={
        "values_a": In(default_value=[]),
        "values_b": In(default_value=[]),
    },
)
def combine_lists(context, values_a, values_b):
    <http://context.log.info|context.log.info>(f"Combining collected values: A: {values_a}, B: {values_b}")
Another update: I got the desired behavior by dropping the "split_item" conditional op and instead having "make_list" produce multiple dynamic outputs (see attached). I'm still puzzled as to why the conditional branching pattern didn't work inside a map/collect, but I've at least got something that addresses my use case.
example_job.py
s
Hey Andrew - thanks for reporting this. I've filed an issue to track this: https://github.com/dagster-io/dagster/issues/8574.
2 Views
Open in Slack
PreviousNext
Powered by