https://dagster.io/ logo
#ask-community
Title
# ask-community
k

Kirill Yakunin

02/13/2023, 3:06 PM
Hello! I have a question regarding Kubernetes deployment. I've deployed Dagster according to this guide: https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm Now, I want my configure node which will execute my asset code:
Copy code
@asset(
    op_tags={
        'dagster-k8s/config': {
            'container_config': {
                'image': BASIC_EXECUTOR_IMAGE,
                "resources": {
                    "requests": {"cpu": "50m", "memory": "2Gi"},
                }
            },
            "pod_spec_config": {
                "nodeSelector": {
                    "role": "..."
                }
            }
        },
    }
)
def fake_init_data(context):
    <http://context.log.info|context.log.info>("Test log!")
    return [
        {"date": "2022-01-01", "value": 5},
        {"date": "2022-01-01", "value": 5},
        {"date": "2022-01-01", "value": 5},
        {"date": "2022-01-01", "value": 5},
        {"date": "2022-01-01", "value": 5},
        {"date": "2022-01-01", "value": 5},
        {"date": "2022-01-01", "value": 5},
    ]
When I try to run it (materialize the asset), it actually get ran using my code-deployment image, and not the one specified, and nodeSelector/resources also are ignored. If I try
Copy code
fake_init_data_job = define_asset_job(
    "fake_init_data_job", AssetSelection.keys(['fake_init_data']),
    tags={
        'dagster-k8s/config': {
            'container_config': {
                'image': BASIC_EXECUTOR_IMAGE,
                "resources": {
                    "requests": {"cpu": "50m", "memory": "2"Gi},
                }
            },
            "pod_spec_config": {
                "nodeSelector": {
                    "role": "...."
                }
            }
        },
    }
)
and then run the job, the job gets executed correctly (with correct image, on correct node). Any ideas on how to make asset materialization take the op_tags into account? Would appreciate any help 🙏
Solved! You can set executor property on you Definitions object, or you can pass default_executor_def argument to your @repository like that:
Copy code
@repository(default_executor_def=k8s_job_executor)
Then, all of assets and jobs will be executed by k8s_job_executor by default, and op_tags will be taken into account.
🌈 1
6 Views