19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Hey folks! I am looking into creating a k8s job co...
# deployment-kubernetesj
Javier Llorente Mañas
02/01/2022, 4:10 PMHey folks! I am looking into creating a k8s job configuration for a job using Volumes mounting a secret, how could I set these objects I am searching through the source code, but I do not have a clear picture of should I define volumes?
d
daniel
02/01/2022, 5:07 PMHi javier, you can set volumes using the volumes/volumeMounts keys in the helm chart here: https://github.com/dagster-io/dagster/blob/master/helm/dagster/values.yaml#L444-L463 - you may need to set them on both the run launcher and in the user-code-deployments section (the former is used when the run is launched, the latter is used when serving metadata about your jobs to dagit)
j
Javier Llorente Mañas
02/01/2022, 6:29 PMOk an there is any possibility to set up this volumes as part of the job generation in python code ?
d
daniel
02/01/2022, 6:31 PMits possible to add tags to your jobs in python that configure the k8s job - you could use that to configure the container that is spun up when the job is launched and add the volumes there: https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#job-or-op-kubernetes-configuration
j
Javier Llorente Mañas
02/01/2022, 6:32 PMOk I see thanks for you quick response
Javier Llorente Mañas
02/01/2022, 6:32 PMAnd do you know an example or at least which is the model use by the tags, it follows the standard k8s model?
Javier Llorente Mañas
02/01/2022, 6:32 PMAlso I could use this tags in a job level aswell?
d
daniel
02/01/2022, 6:36 PMyeah, they should be passed into a V1Container, V1PodSpec, etc. : https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-k8s/dagster_k8s/job.py#L556-L586 - and you should put them on the Dagster @job decorater, yeah (it won't actually work if you set it on the @op ) unless you are using a certain executor that runs each op in its own pod
j
Javier Llorente Mañas
02/01/2022, 7:16 PMlooking through the code I can imagine that something like this could work
Javier Llorente Mañas
02/01/2022, 7:16 PM Copy code
@job(tags={'dagster-k8s/config': {'container_config': {'resources': {'requests': { 'cpu': '1', 'memory': '500Mi' },}},
'job_config':{'volume':[{'name': 'train_dir', 'mountPath': '/train_dir'}], 'volume_mounts':[{'name': 'train_dir', 'mountPath': '/train_dir'}]}}})
def train_model_job():
passd
daniel
02/01/2022, 7:16 PMI think it would be
volumesvolumeMountsdaniel
02/01/2022, 7:17 PMor, sorry, you may be right
daniel
02/01/2022, 7:17 PMvolume_mounts
daniel
02/01/2022, 7:17 PMmy mistake
2 Views