The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

My team is having trouble implementing `tag_concurrency_limits` when deploying on *k8s*
Our run coordinator includes the following:
```run_coordinator:
  module: dagster._core.run_coordinator
  class: QueuedRunCoordinator
  config:
    max_concurrent_runs: 2
    tag_concurrency_limits:
    - key: catalog
      limit: 4
      value: iceberg```
The relevant `ops` are decorated with `@op(tags={"catalog": "iceberg"})`
Yet, more than 4 of these ops will process concurrently. The ops are being spawned by a `yield DynamicOutput(...)`. Is this a special case where I need to do something differently?
We'd like to limit the number of spawned ops that access one of our databases.

Hi Chris - those tag limits only apply to jobs/runs, not to individual ops. You can also apply tag limits to ops within a single run, but we don’t yet have a way to globally limit the number of ops with a certain tag across all runs

Or, actually - on k8s specifically there is a way to specify limits like this with celery: <https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm-advanced|https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm-advanced>

<@U016C4E5CP8>, thanks for the help! I'll be working on this again in about 12 hours with someone else on our team who does the deploying.