My team is having trouble implementing

tag_concurrency_limits

when deploying on k8s Our run coordinator includes the following:

run_coordinator:
  module: dagster._core.run_coordinator
  class: QueuedRunCoordinator
  config:
    max_concurrent_runs: 2
    tag_concurrency_limits:
    - key: catalog
      limit: 4
      value: iceberg

The relevant

ops

are decorated with

@op(tags={"catalog": "iceberg"})

Yet, more than 4 of these ops will process concurrently. The ops are being spawned by a

yield DynamicOutput(...)

. Is this a special case where I need to do something differently? We'd like to limit the number of spawned ops that access one of our databases.

Hi Chris - those tag limits only apply to jobs/runs, not to individual ops. You can also apply tag limits to ops within a single run, but we don’t yet have a way to globally limit the number of ops with a certain tag across all runs

@daniel, thanks for the help! I'll be working on this again in about 12 hours with someone else on our team who does the deploying.