https://dagster.io/ logo
#deployment-kubernetes
Title
# deployment-kubernetes
a

Abhishek Agrawal

04/27/2023, 6:34 AM
Hello, we have Dagster running on OSS (GKE Autopilot) and we are running into some trouble with docker images. We are using a custom tag and the
pullPolicy
is set to
Always
. When we do a docker push, dagster is not pulling the latest image. Has anyone faced this before? Any fix?
a

alex

04/27/2023, 3:04 PM
the custom tag is different every update or are you trying to do updates in place? If you are doing updates in to the same docker tag you need to do something to force all the pods to restart so that they will pull the new image
a

Abhishek Agrawal

04/27/2023, 9:46 PM
It's the same docker tag. We kind of guessed this. So, how does it work in the Hybrid Cloud setting? Does the agent we install make sure that pods are restarted? Any pointers what's the best way to restart all the pods?
a

alex

04/27/2023, 9:59 PM
Using a scheme where the tag changes when the code changes is the best way i would say. Hybrid Cloud would suffer the same problem if you are trying to update a tag in place.
a

Abhishek Agrawal

04/28/2023, 1:54 AM
I think on Cloud I was using
latest
tag and it would automatically take new image without me restarting pods..
a

alex

04/28/2023, 2:17 PM
ah ok, it depends on what how you are interacting with Cloud when you update the
latest
tag. Updating other pieces of metadata or explicitly asking for a redeploy would work.
s

Simon Frid

04/28/2023, 2:44 PM
i often recommend something like
$GIT_COMMIT_SHA-$TIMESTAMP
for an image, so that way 2 things happen. 1. you always know which commit is associated with your image 2. the change in the timestamp always re-triggers a pull
1
1
a

Abhishek Agrawal

05/01/2023, 12:19 AM
Thanks @alex and @Simon Frid! We did something like this..
29 Views