@Scout setup max concurrent pod run within each run to 30 using this configuration : runLauncher: type: K8sRunLauncher config: k8sRunLauncher: runK8sConfig: containerConfig: # raw config for the pod's main container (job container) resources: limits: cpu: 2 memory: 8Gi ephemeral-storage: 10Gi

The configuration snippet you've provided sets resource limits for the main container of a job when using the

K8sRunLauncher

. However, it does not directly set a limit on the maximum number of concurrent pod runs within each run. The resource limits specified in the

containerConfig

section (

cpu

,

memory

, and

ephemeral-storage

) are constraints on the resources available to each individual pod, not the concurrency of pod execution. To limit the maximum number of concurrent pod runs within each run, you would need to configure concurrency controls at the job or asset level. The documentation provides information on how to set concurrency limits using

tag_concurrency_limits

, which can be applied to limit the number of ops with certain tags that can execute at once within a single run. Here is an example from the documentation on how to use `tag_concurrency_limits`:

yaml run_queue:   tag_concurrency_limits:     - key: "my-concurrency-limiter"       value: "value1"       limit: 30

In this example, the

tag_concurrency_limits

configuration is set in the

dagster.yaml

file, which is the configuration file for the Dagster instance. It specifies that for any op tagged with

my-concurrency-limiter: value1

, at most 30 instances of such ops can run concurrently. To apply this to your scenario, you would need to ensure that the relevant ops or assets in your job are tagged appropriately and that the

dagster.yaml

file includes the

tag_concurrency_limits

configuration with the desired concurrency limit. Please note that the exact implementation details may vary based on your specific use case and the version of Dagster you are using. Always refer to the latest Dagster documentation for the most accurate and up-to-date guidance on configuring concurrency limits.