19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Hi, our code deployment suddenly failed and starte...
# dagster-plusa
Abhishek Agrawal
03/07/2023, 4:52 AMHi, our code deployment suddenly failed and started giving this DagsterUserCodeUnreachableError error. It was working fine until an hour ago and there have been no updates to the docker image. The agent pod is giving this error in the logs.
t
Tobias Pankrath
03/07/2023, 7:56 AMMight be the same as my problem from the support channel?
a
Abhishek Agrawal
03/07/2023, 8:11 AMMaybe.. mine was working and then all of a sudden this error started coming
d
daniel
03/07/2023, 1:25 PMIf you run 'kubectl describe` on the pod with 'spina' in the name that it's trying to connect to there, is there any indication why it failed to start up?
a
Abhishek Agrawal
03/07/2023, 1:53 PMThe logs don't show any issue at all. Just normal ones. I will run describe and let you know here.
d
daniel
03/07/2023, 1:55 PMIs the pod possibly taking more than 3 minutes to start up? The timeout that it's using there is configurable in the agent helm chart
daniel
03/07/2023, 1:56 PMif it's expected for it to take more than 3 minutes for your code to load after the pod starts up, you can increase this
serverProcessStartupTimeouta
Abhishek Agrawal
03/07/2023, 2:13 PMLoading code means pulling docker image? Actually it doesn't take that long to pull the image, maybe 30-40s.
d
daniel
03/07/2023, 2:13 PMI was referring to importing your Python / loading your Dagster definitions in code
daniel
03/07/2023, 2:14 PMIf you have logs or a 'kubectl describe' output that might give a sense of what was happening during those 3 minutes before it timed out, we can take a look
👍 1
a
Abhishek Agrawal
03/07/2023, 2:18 PMGot it. I think I know what you mean now and it might be the reason. We are running a loop which is taking really long. I will try to push some code changes and see. Thanks!
3 Views