https://dagster.io/ logo
Title
r

Rafael Nogales

01/18/2023, 1:53 PM
Hello guys. I'm trying to limit the concurrency. What I have done: I have created a new tag
cross_job_concurrency
and I put a value 9 (for example) to override the default value of 10 When I launch the backfill I add this custom tag. But when I run the backfill it still having 10 jobs starting Some suggestion?
dagsterDaemon:
      runCoordinator:
        enabled: true
        type: QueuedRunCoordinator
        config:
          queuedRunCoordinator:
            maxConcurrentRuns: 100
            tagConcurrencyLimits:
            - key: dagster/backfill
              value:
                applyLimitPerUniqueValue: true
              limit: 10
            - key: cross_job_concurrency
              value: 9
              limit: 9
....
c

chris

01/18/2023, 7:42 PM
I think you also may need to set the value for the tag to match what’s in your
dagster.yaml
, so in this case
9