The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Is there a suggested developer workflow for deploying workspaces to dagster k8s?

Hi <@U01F1CL1CLX> please check out our k8s deployment docs: <https://docs.dagster.io/deploying/kubernetes>

Hi <@UULA0R2LV> I have everything deployed. The helm chart values let you select user code and repos but once that is done how would iterate on those repos?

Ah got it! Did you mean deploying multiple repos / user code, i.e. multiple User Code Deployments? or the dev workflow of iterating the user code after you’ve deploy it?

exactly. looking to write a bunch of new stuff and "deploy" those workloads

hey Adrian - our workflow around this is that we created a CI workflow to `helm upgrade` our release whenever we push changes to our repository holding our dagster pipeline code

I have microk8s w/ the basic k8s deployment going but I want to ship solids and pipelines as workloads

<@U01AV4K3W14> ah I see. I'm using kustomize to generate my manifests for different environments. It looks like the only way to do this would be to re apply those manifests?  Hard part is I keep my infra in a different project then my workloads

If I'm understanding this correctly, if my repo are shipped to my docker registry and the deployment is set to pull `tag: latest` as long as the deployment config is not changing when run it should pickup the changes?

do you have the source for this example? <https://hub.docker.com/r/dagster/k8s-example>

I’m not familiar with kustomize, but I’m assuming it works like Helm. In Helm, upgrades are triggered are diffing each manifests - for the manifests that have changed, Helm will redeploy those

if you set `tag:latest` and nothing else has changed, by itself it will not pickup the changes

you’ll need to add some dynamic part in the manifest: similar to <https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments>

that docker link is deprecated - we have <https://hub.docker.com/r/dagster/user-code-example> instead! it’s located here <https://sourcegraph.com/github.com/dagster-io/dagster/-/tree/python_modules/automation/automation/docker/images/k8s-example>

ah, the values.yml allows you to specify annotation too

we’ve used an annotation with the current timestamp of the deployment to force a re-deployment

yeah - in our case, we don’t use `tag:latest`, we tag images by their git hash so each deployment manifest will be different (if indeed the pipeline code changed)

I'll have to pull the helm chart and separate the configmaps to the other repo that has the Dockerfile and solids and use Skaffold