https://dagster.io/ logo
Title
n

Nick Dellosa

11/19/2021, 5:34 PM
I see in the docs that in the newer versions of Dagster, you can specify config at the job and graph level instead of just at the op level. Does anyone have an example of what this looks like?
👀 1
c

chris

11/19/2021, 6:45 PM
Let's say I have the following op:
@op(config_schema=str)
def my_op(context):
    ...
Then the following graph which contains the op can be configured like so:
@graph(config={"my_op": {"config": "bar"}})
def my_graph():
    my_op()
For a job containing this op:
@job(config={"ops": {"my_op": {"config": "bar"}}})
def my_job():
    my_op()
n

Nick Dellosa

11/19/2021, 8:02 PM
@chris Is it possible to specify a config value for a graph or job once and then have it propagate to all of the ops?
c

chris

11/19/2021, 8:03 PM
What do you mean by propagate in this context? As in, you provide one piece of config to the graph, and that config piece is passed to every single op?
n

Nick Dellosa

11/19/2021, 8:03 PM
yeah exactly
c

chris

11/19/2021, 8:06 PM
I think config mapping is what you want in that case. Let me know if you have further questions after checking out this doc.
👍 1
s

sandy

11/19/2021, 10:41 PM
another way to make config available to every op inside a job is with a resource: https://docs.dagster.io/concepts/configuration/config-schema#passing-configuration-to-multiple-ops-in-a-job
2