https://dagster.io/ logo
Title
m

Mykola Palamarchuk

01/13/2022, 4:07 PM
I've also faced another problem: Dagit doesn't reload
@repository
when I build new user-deployments image and upgrade the helm chart. But the pod is updated, so Dagit can't run it, but it doesn't see the changes. I have to uninstall and install the chart again to make it work. I'm using "latest" tag on my image. Could that be a reason?
j

johann

01/13/2022, 4:08 PM
Hi Nick, have you tried reloading the workspace from Dagit?
m

Mykola Palamarchuk

01/13/2022, 4:12 PM
Yes, and that doesn't update the repository. It works only when I do
helm uninstall
and then
helm upgrade --install
again for my user deployments.
j

johann

01/13/2022, 4:13 PM
Do you have an
imagePullPolicy
set for your user deployments?
You’ll want to guarantee it’s Always if you’re using latest
m

Mykola Palamarchuk

01/13/2022, 4:25 PM
The policy is "Always". The pod is actually updated, but Dagit can't see that somehow. When the job is launched it shows errors from the updated pod (e.g. it can't find "old" job names).
j

johann

01/13/2022, 4:28 PM
Hmm, that behavior is consistent with either the deployments not pulling the new image or not reloading the workspace after they’ve finished upgrading
I suspect your helm upgrade isn’t triggering the deployments to pull a new image
Could you try it out not using latest
m

Mykola Palamarchuk

01/13/2022, 4:33 PM
Will do it later and let you know.
Using exact tag works.
j

johann

01/13/2022, 5:17 PM
You could take a look at https://github.com/helm/helm/issues/5696, I think they ran in to a similar issue. In general it’s easiest to just avoid using
latest
, but there are some workarounds that will force k8s to do a upgrade to the deployment which would pick up the new image