https://dagster.io/ logo
Title
d

Dylan Hunt

09/14/2021, 2:42 PM
I think the document here https://docs.dagster.io/concepts/solids-pipelines/pipelines#pipeline-dsl is wrong. How can we pass same solid to itself as input while the actual add_one needs a input. Can someone please explain? Even if I run it using dagit, it shows error for A
m

max

09/14/2021, 3:59 PM
the root inputs to the pipeline would presumably be provided through config
❤️ 1
d

Dylan Hunt

09/14/2021, 4:32 PM
@max So can we pass config also in yaml? or only through the config. Also is there other way to construct the pipeline similar to this? like json or dynamically generated yaml instead of static file?
m

max

09/14/2021, 4:35 PM
if you wanted to define config in yaml you could use the
configured
API
and yes, in theory there is no reason you couldn't write some code to translate JSON or any other source format into a pipeline definition
the document you linked just shows an example of how you might do that
❤️ 1
d

Dylan Hunt

09/14/2021, 4:40 PM
Thanks much for the quick response @max
But configured API will be passed during the execution, can we add it somehow during the construct pipeline method?
m

max

09/14/2021, 4:47 PM
hm, i'm not sure i follow --
configured
lets you set config at definition time, not at execution time https://docs.dagster.io/concepts/configuration/configured#configured-api
d

Dylan Hunt

09/14/2021, 4:50 PM
Okay, will check, I'm new to dagster and trying to dynamically generate both solids and pipeline. Appreciate your help and time
👍 1
m

max

09/14/2021, 4:54 PM
lmk if you hit any snags
❤️ 1
d

Dylan Hunt

09/14/2021, 4:56 PM
Sure, @max