https://dagster.io/ logo
#deployment-kubernetes
Title
# deployment-kubernetes
b

Binoy Shah

05/26/2022, 5:59 PM
Is there a example of user-deployments chart based deployment for kubernetes ? Wanted to check how the docker image is to be built and deployed,
🤖 1
s

sean

05/27/2022, 12:36 PM
cc @rex @johann
For how to deploy it, you’d just want to push to some image registry (Dockerhub, ECR, etc.)
b

Binoy Shah

05/27/2022, 2:57 PM
so one thing was a bit fuzzy for me. Each
user-deployment
docker image could have more than one job pipeline ( python dags ), and from that python dag, i create either kubernetes runner to deploy kubernetes or celery+kubernetes jobs ?
that will go to kubernetes API server, and the Kube API server will create Kubernetes Job or a Kubernetes+Celery Job ?
j

johann

05/27/2022, 5:06 PM
each
user-deployment
docker image could have more than one job pipeline
yep, each image holds what we call a
Repository Location
, which can hold multiple `@repository`s, which hold multiple `@job`s.
We’ll use the image to get metadata about the structure of the Dag to display it in the UI, and we’ll use that image when we launch K8s Jobs
b

Binoy Shah

05/27/2022, 5:08 PM
what about celeryk8s jobs>? doe it need a celery worker running, or will the dagster’s “operator” start a celery docker container and send task message to RMQ ?
j

johann

05/27/2022, 5:09 PM
The helm chart spins up celery workers if you specify the CeleryK8sRunLauncher
b

Binoy Shah

05/27/2022, 5:10 PM
perfect.. thank you so much for taking time to answer the questions.. documentation is extensive, but it needs a kubernetes quickstart kind of thing too 🙂
👍 1