Join Slack
Powered by
hey everyone! Quick question: how can I reuse outp...
# announcements
e
eamag
09/27/2019, 11:59 AM
hey everyone! Quick question: how can I reuse output of the solid, should I just put it twice in the pipeline? Will it be calculated twice or not?
eamag
09/27/2019, 12:03 PM
So I want to download and transform the data in one solid, then split pipeline in branches, calculate in parallel and combine the output and write it
p
Philipp G
09/27/2019, 1:12 PM
Sounds like the example that is explained in the tutorial
https://dagster.readthedocs.io/en/0.5.9/sections/learn/tutorial/actual_dag.html
Parallel computing is not a part of this small example, but you will learn how to produce multiple outputs and how to combine them again.
e
eamag
09/27/2019, 1:17 PM
oh thanks, missed multiple times somehow 😅
👍 1
m
max
09/27/2019, 4:15 PM
you can have multiple solids depend on the same output 🙂
max
09/27/2019, 4:15 PM
it won't be calculated twice
2
Views
Open in Slack
Previous
Next