Is there a suggested developer workflow for deploying worksp

Question

Is there a suggested developer workflow for deploying workspaces to dagster k8s

yuhan · Answer

Hi < Adrian> please check out our k8s deployment docs

Adrian · Answer

Hi < yuhan> I have everything deployed The helm chart values let you select user code and repos but once that is done how would iterate on those repos

yuhan · Answer

Ah got it Did you mean deploying multiple repos user code i e multiple User Code Deployments or the dev workflow of iterating the user code after you ve deploy it

yuhan · Answer

cc < rex>

Adrian · Answer

exactly looking to write a bunch of new stuff and deploy those workloads

rex · Answer

hey Adrian our workflow around this is that we created a CI workflow to `helm upgrade` our release whenever we push changes to our repository holding our dagster pipeline code

Adrian · Answer

I have microk8s w the basic k8s deployment going but I want to ship solids and pipelines as workloads

Adrian · Answer

< rex> ah I see I m using kustomize to generate my manifests for different environments It looks like the only way to do this would be to re apply those manifests Hard part is I keep my infra in a different project then my workloads

Adrian · Answer

If I m understanding this correctly if my repo are shipped to my docker registry and the deployment is set to pull `tag latest` as long as the deployment config is not changing when run it should pickup the changes

Adrian · Answer

do you have the source for this example <https hub docker com r dagster k8s example>

rex · Answer

I m not familiar with kustomize but I m assuming it works like Helm In Helm upgrades are triggered are diffing each manifests for the manifests that have changed Helm will redeploy those

rex · Answer

if you set `tag latest` and nothing else has changed by itself it will not pickup the changes

rex · Answer

you ll need to add some dynamic part in the manifest similar to <https helm sh docs howto charts tips and tricks automatically roll deployments>

rex · Answer

that docker link is deprecated we have instead it s located here

Adrian · Answer

ah the values yml allows you to specify annotation too

Peter Stratton · Answer

we ve used an annotation with the current timestamp of the deployment to force a re deployment

rex · Answer

yeah in our case we don t use `tag latest` we tag images by their git hash so each deployment manifest will be different if indeed the pipeline code changed

Adrian · Answer

okay great I m going to try this out

Adrian · Answer

I ll have to pull the helm chart and separate the configmaps to the other repo that has the Dockerfile and solids and use Skaffold