https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Anoop Sharma

08/03/2022, 5:19 PM
How can I set up the executor config for all the jobs in the all the repositories? I have want to set up max concurrent op limit for all the jobs to 2. I have come across this yaml config:
Copy code
execution:
  config:
    multiprocess:
      max_concurrent: 2
but where do I put this config. Does it go in dagster.yaml? PS: I have tried putting it in dagster.yaml but it doesn't seem to be working.
🤖 1
o

owen

08/03/2022, 5:55 PM
hi @Anoop Sharma! That config is meant to be applied to the run config of a particular job. That's in yaml format (so that it can be copy-pasted into the Launchpad in dagit). In python code, that would look like
{"execution": {"config": {"multiprocess": {"max_concurrent": 2}}}}
. Unfortunately, this would need to be applied to each individual job (i.e.
@job(config={...})
) -- there's not currently a way to apply the same executor config to all jobs at once. To be clear, this would set the max concurrent ops for each job to 2, but if there were multiple jobs running at once, then more than 2 ops total would be running at once (max 2 per job)
a

Anoop Sharma

08/03/2022, 6:08 PM
Got it. Thanks for the clarification @owen.
8 Views