I'm running the dagster helm chart with resource limits and trying to override the limits for a single job. However, it doesn't seem to be working (the job container get's the default resource contraints set):

@op(
    # This task is extremely inefficient and requires a lot of memory
    tags={
        "dagster-k8s/config": {
            "container_config": {
                "resources": {
                    "requests": {"cpu": "500m", "memory": "8Gi"},
                    "limits": {"cpu": "2", "memory": "12Gi"},
                }
            },
        }
    }
)
def run_op(context: OpExecutionContext) -> None:
    ...

my_asset = AssetsDefinition.from_op(
    run_op,
)

The run launcher config is set to:

run_launcher:
  module: dagster_k8s
  class: K8sRunLauncher
  config:
    ...    
    resources:
      limits:
        cpu: '1'
        memory: 2Gi
      requests:
        cpu: 500m
        memory: 1Gi
    ...

Any ideas why this wouldn't work?

you should put this tag on the job, not the op

the job has multiple ops, and I don't really want to use these resources for every op of the job. kind of like it says in the documentation:

its a little confusing, but there are two separate things at play here - the run launcher (where the run happens) and the executor (where each individual op/step happens). Using the K8sRunLauncher will ensure that the run happens in a k8s pod, but doesn't imply that every op will happen in its own pod. The default behavior if you don't specify an executor is for each op to execute in its own process, but still all in the same kubernetes pod

Hi @Davis Kirkendall, again you ^^ I had AGAIN the exact same issue just some days ago, Have a look here: https://github.com/dagster-io/dagster/issues/8980. Also related here https://github.com/dagster-io/dagster/issues/8405