The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Is there a example of user-deployments chart based deployment for kubernetes ? Wanted to check how the docker image is to be built and deployed,

Here’s an example docker file: <https://github.com/dagster-io/dagster/blob/master/python_modules/automation/automation/docker/images/user-code-example/Dockerfile>

For how to deploy it, you’d just want to push to some image registry (Dockerhub, ECR, etc.)

so one thing was a bit fuzzy for me. Each `user-deployment` docker image could have more than one job pipeline ( python dags ), and from that python dag, i create either kubernetes runner to deploy kubernetes or celery+kubernetes jobs ?

that will go to kubernetes API server, and the Kube API server will create Kubernetes Job or a Kubernetes+Celery Job ?

&gt; each `user-deployment` docker image could have more than one job pipeline
yep, each image holds what we call a `Repository Location`, which can hold multiple `@repository`s, which hold multiple `@job`s.

We’ll use the image to get metadata about the structure of the Dag to display it in the UI, and we’ll use that image when we launch K8s Jobs

what about celeryk8s  jobs&gt;? doe it need a celery worker running, or will the dagster’s “operator” start a celery docker container and send task message to RMQ ?

The helm chart spins up celery workers if you specify the CeleryK8sRunLauncher

perfect.. thank you so much for taking time to answer the questions.. documentation is extensive, but it needs a kubernetes quickstart kind of thing too  :slightly_smiling_face: