https://dagster.io/ logo
Title
m

Martin Laurent

02/28/2022, 10:24 AM
Hey everyone! First message here 👋 I'm trying to build a subgraph that would either execute an expensive op if a check passes, or skip it and return the input if the check fails (input/outputs are file paths). I've been looking at "Conditional Branching" but I'm not sure how to merge the two resulting branches (return the output of the op that was executed. I've also been looking to ExpectationResult which sounded close to what I was doing, but I can't figure out how to access the result at runtime. (I've been looking in the expensive op context)
a

alex

02/28/2022, 4:54 PM
merge the two resulting branches
https://docs.dagster.io/concepts/ops-jobs-graphs/jobs-graphs#fixed-fan-in
looking to ExpectationResult
This object so far is only set-up to communicate to dagster and not really be read back out. I would recommend using your own type as an output if you want to read it in a downstream op
m

Martin Laurent

03/03/2022, 2:29 PM
Hey! Coming back to this "merging the outputs of a conditional branching": I can't use fixed-fan-in because it explicitly says in the doc:
The downstream op executes only if all of the outputs were successfully created by the upstream op.
so any op that uses both output of the conditional branching will never execute because 1 of them has not been executed. Any other potential solution? My use case is depending on some condition I may wan to use one or another algo (op or graph) but both would return an output file. I want to use this output file in subsequent ops
a

alex

03/03/2022, 2:58 PM
Should be clarified in the docs, but a fan-in dep will only skip if all fanned in outputs skipped, so should still run it only some were skipped