https://dagster.io/ logo
e

eamag

09/27/2019, 11:59 AM
hey everyone! Quick question: how can I reuse output of the solid, should I just put it twice in the pipeline? Will it be calculated twice or not?
So I want to download and transform the data in one solid, then split pipeline in branches, calculate in parallel and combine the output and write it
p

Philipp G

09/27/2019, 1:12 PM
Sounds like the example that is explained in the tutorial https://dagster.readthedocs.io/en/0.5.9/sections/learn/tutorial/actual_dag.html Parallel computing is not a part of this small example, but you will learn how to produce multiple outputs and how to combine them again.
e

eamag

09/27/2019, 1:17 PM
oh thanks, missed multiple times somehow 😅
👍 1
m

max

09/27/2019, 4:15 PM
you can have multiple solids depend on the same output 🙂
it won't be calculated twice