Hi #dagster-kubernetes! I am deploying dagster using the helm chart. I want to add some resources to my k8sRunLauncher, as otherwise my jobs get OOMKilled. I am using the following

values.yaml

for the runlauncher part:

runLauncher:
  type: K8sRunLauncher
  config:
    k8sRunLauncher:
      resources:
        requests: 
          cpu: 250m
          memory: 256Mi
        limits:
          cpu: 1
          memory: 4Gi
      envVars:
        - GOOGLE_APPLICATION_CREDENTIALS=/root/.config/gcloud/key.json
      volumes:
        - name: gcloud-secrets
          secret:
            secretName: gcloud
      volumeMounts:
        - name: gcloud-secrets
          mountPath: /root/.config/gcloud

The

dagster-instance

configmap part looks good:

run_launcher:
  module: dagster_k8s
  class: K8sRunLauncher
  config:
    load_incluster_config: true
    job_namespace: gww
    image_pull_policy: Always
    service_account_name: dagster
    dagster_home: "/opt/dagster/dagster_home"
    instance_config_map: "dagster-instance"
    postgres_password_secret: "dagster-postgresql-secret"
    env_vars:
      - GOOGLE_APPLICATION_CREDENTIALS=/root/.config/gcloud/key.json
    volume_mounts:
      - mountPath: /root/.config/gcloud
        name: gcloud-secrets
    volumes:
      - name: gcloud-secrets
        secret:
          secretName: gcloud
    resources:
      limits:
        cpu: 1
        memory: 4Gi
      requests:
        cpu: 250m
        memory: 256Mi

Still, when describing a pod lauched from the dagit UI, I am not seeing my resource requirements back, while all the other settings (e.g. volumes/mounts, env vars etc.) are being set on the pod. I am out of ideas. Anyone has encountered this before? Thanks in advance!

Hi @Jaap Langemeijer, yeah this is tricky. Currently there’s two options: • disable

includeConfigInLaunchedRuns

, which configures the overriding. https://github.com/dagster-io/dagster/blob/master/helm/dagster/values.yaml#L268-L271 The downside is that things like secrets etc. will also not be copied over • override the override: you can specify resources per job using tags: https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment

Created issue at: https://github.com/dagster-io/dagster/issues/8749

@johann thanks for the clear answer!