Hi everyone, is it possible to set job-level concurrency limit? I know that I can set:

execution:
  config:
    multiprocess:
      max_concurrent: 2

so that within one job, only two sub-task/ops run at the same time But, I am more interested in e.g. triggering 1000 job-runs/backfills, but only 2 of those are running, and 998 on queue until one of those 2 completes. I know that there are also Dagster-level job-limit, but I don't want e.g. those 1000 jobs clogging up/preventing other existing jobs

https://github.com/dagster-io/dagster/discussions/12251 This thread might be helpful

Thanks! let me check...

hey @Chang Hai Bin I think tag-based run concurrency limits will solve this for you. A run is just an execution of a job, so you would just add a specific tag to the job you want to limit and configure what that limit should be. Then dagster will ensure that no more than that many runs (ie executions of that job) are happening at once