Hey guys, I'm trying to use the

k8s_job_executor

but my GCP job will get stuck at "Pods are pending". This is my run config:

Config
execution:
  k8s: null
solids:
  solid_get_company_info:
    inputs:
      company: myCompany

And this is my ModeDefinition:

default_mode = ModeDefinition(
    name="default",
    resource_defs={
        "io_manager": fs_io_manager,
        "datadog": configured_datadog,
    },
    executor_defs=default_executors + [k8s_job_executor],
)

And my tags:

default_k8s_tag = {
    "dagster-k8s/config": {
        "container_config": {
            "env": [
                {
                    "name": "DD_AGENT_HOST",
                    "valueFrom": {"fieldRef": {"fieldPath": "status.hostIP"}},
                }
            ],
            "resources": {
                "requests": {"cpu": "125m", "memory": "250Mi"},
                "limits": {"cpu": "500m", "memory": "2Gi"},
            },
        },
        "pod_spec_config": {
            "affinity": {
                "nodeAffinity": {
                    "requiredDuringSchedulingIgnoredDuringExecution": {
                        "nodeSelectorTerms": [
                            {
                                "matchExpressions": [
                                    {
                                        "key": "type",
                                        "operator": "In",
                                        "values": [
                                            "data-worker",
                                        ],
                                    }
                                ]
                            }
                        ]
                    }
                }
            },
        },
    },
}

My runLauncher def in values.yaml:

runLauncher:
  type: K8sRunLauncher
  config:
    k8sRunLauncher:
      jobNamespace: data
      envConfigMaps:
        - name: dagster-release-dagster-user-deployments-myservice-user-env
        - name: my-secrets

I am using Dagster 0.12.15, any ideas?

Hi, are there any logs or output of kubectl describe from the k8s job/pod for the step that might explain how it got stuck?

kubectl describe pod

might be useful to explain why the pod is in a pending status