https://dagster.io/ logo
#ask-community
Title
# ask-community
s

Shahab Tasharrofi

08/28/2023, 10:49 PM
Hi there! What is the best way to model non-strict dependencies? That is, I have an operation
op
that takes the result of some other (potentially complex) subgraphs (let’s say three subgraphs
a
,
b
and
c
) and combines them into one result. So,
op
only fails if all of the subgraphs
a
,
b
and
c
fail. But if, for example,
a
fails but
b
and
c
both return a result,
op
still succeeds and returns a result that combines the results of
b
and
c
. Now, since each one of
a
,
b
and
c
in this example can be complex graphs which can fail at different stages of the computation, I need to model
op
so that it waits until, for each of
a
,
b
and
c
, either the result is ready or it is determined that they cannot have a result and then
op
is run and the available results for
a
,
b
and
c
are passed to it. The potential use case for this is. e.g., when you want to do ensemble learning and combine predictors of different estimations for the same problem. So, if one predictor fails and cannot provide a value, it should not stop the whole prediction process.
s

sandy

08/29/2023, 4:10 PM
s

Shahab Tasharrofi

08/30/2023, 12:26 AM
Thanks @sandy ! Answered on Github.