Hi there! What is the best way to model non-strict dependencies? That is, I have an operation

op

that takes the result of some other (potentially complex) subgraphs (let’s say three subgraphs

a

,

b

and

c

) and combines them into one result. So,

op

only fails if all of the subgraphs

a

,

b

and

c

fail. But if, for example,

a

fails but

b

and

c

both return a result,

op

still succeeds and returns a result that combines the results of

b

and

c

. Now, since each one of

a

,

b

and

c

in this example can be complex graphs which can fail at different stages of the computation, I need to model

op

so that it waits until, for each of

a

,

b

and

c

, either the result is ready or it is determined that they cannot have a result and then

op

is run and the available results for

a

,

b

and

c

are passed to it. The potential use case for this is. e.g., when you want to do ensemble learning and combine predictors of different estimations for the same problem. So, if one predictor fails and cannot provide a value, it should not stop the whole prediction process.

Hey Shahab - just wrote this up: https://github.com/dagster-io/dagster/discussions/16166

Thanks @sandy ! Answered on Github.