19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Channels
#dagster-support
Title
# dagster-support
p
Pieter Custers
12/19/2022, 5:41 PMHey I’m running into the issue that the image used by the run worker is not the same as the one of the repo. This must be a bug. Running on Dagster 1.1.4, both user code and system.
This is what the UI shows
From the description of the grpc pod
From the pod of the run worker, note
20602065I did a rollout restart of the deployment of this repo, and that actually seemed to work for a bit. However after a bit it took the old image again
From the pod description right after the rollout restart
d
daniel
12/19/2022, 5:48 PMHey Pieter - somebody reported something similar here and I think we traced it down to an issue that was fixed in 1.1.6 https://dagster.slack.com/archives/C01U954MEER/p1671215044161139?thread_ts=1671204360.330419&channel=C01U954MEER&message_ts=1671215044.161139
Or by manually reloading in dagit after updating the code
p
Pieter Custers
12/19/2022, 5:48 PMHmm I did a manual reload actually
But let me upgrade to 1.1.6, hope that solves it
Thanks @daniel!
And hope you get better soon 🍎
🙏 1
Actually I do find the error in the Dagit pod you mention in that thread (
AttributeError: 'function' object has no attribute 'handle_event'Fyi. Anyway, I’ll upgrade
d
daniel
12/19/2022, 5:52 PMI’m having trouble mapping that to the specific symptoms that you’re reporting, but yeah it definitely can’t hurt to try upgrading
p
Pieter Custers
12/19/2022, 5:53 PMI refer to this
d
daniel
12/19/2022, 5:54 PMOh yeah I just mean I don’t have an explanation for why the problem would still be happening after reloading in dagit
p
Pieter Custers
12/19/2022, 5:54 PMAaaaahaa
Hmm okay I’ll let you know if the problem persists in 1.1.7
d
daniel
12/19/2022, 5:59 PMAre you using the run queue?
p
Pieter Custers
12/20/2022, 7:31 AMMorning
Are you using the run queue?Yes
Upgraded to 1.1.7 and redeployed the concerning repo. The problem has disappeared. But as I don’t know the root cause I can’t say if the bug is solved so it might come back 🤷
d
daniel
12/20/2022, 7:32 PMHm, ok - my guess is there’s some downstream effect of the bug that was fixed that I didn’t consider, definitely let us know if it happens again
👍 1
2 Views