https://dagster.io/ logo
Powered by
#dagster-support
Title
# dagster-support
h

Huib Keemink

08/23/2022, 2:09 PM
I’ve been banging my head against the wall for a while now, hopefully I’m just overlooking something stupid 🙂 What’s the best way to have 
k8s_job_op
respect my tolerations/nodeSelector? I’ve tried to give the job the correct tags, tried setting 
job_config
but nothing seems to work
j

johann

08/24/2022, 5:11 PM
Could you share your code?
h

Huib Keemink

08/24/2022, 8:20 PM
Copy code
sync_solcast_forecast_v1 = k8s_job_op.configured(
    {
        "image": "REDACTED",
        "command": [
            "REDACTED"
        ],
        "args": [
            "REDACTED"
        ],
        "image_pull_policy": "IfNotPresent",
        "env_secrets": ["featurebatches-secret"],
        "load_incluster_config": True,
        "job_config": {
            "spec": {
                "template": {
                    "spec": {
                        "node_selector": {
                            "analyticsOnly": True
                        },
                        "tolerations": [
                            {
                                "key": "analyticsOnly",
                                "operator": "Exists",
                                "effect": "NoSchedule"
                            }
                        ]
                    }
                }
            }
        },
        "image_pull_secrets": [
            {"name": 'acr-creds'},
        ],
        "resources": {
            "requests": {
                "cpu": "0.5",
                "memory": "1024Mi"
            },
            "limits": {
                "cpu": "1",
                "memory": "2048Mi"
            },
        }
    },
    name="sync_solcast_forecast_v1",
)
j

johann

08/25/2022, 8:54 PM
And 
kubectl describe job <name>
doesn’t show any of your config? (can get name from the logs in Dagit, 
Creating Kubernetes job <>
)
d

daniel

09/06/2022, 10:04 PM
Hey Huib - I think we were the ones who were overlooking something stupid - https://github.com/dagster-io/dagster/pull/9594 makes some changes to the field formerly known as job_config, after that goes in I believe your changes would go inside job_spec_config and they would now work
h

Huib Keemink

10/25/2022, 1:26 PM
yeah, I got it to work later, forgot to report back here…
2 Views
Powered by