https://dagster.io/ logo
#deployment-kubernetes
Title
# deployment-kubernetes
r

Rubén Lopez Lozoya

05/26/2021, 2:00 PM
Guys I am experiencing some issues with my dagit deployment. I see this error frequently in my GCP logs. Any ideas?
"Liveness probe failed: Get "<http://10.87.6.3:80/dagit_info>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
j

johann

05/26/2021, 2:01 PM
cc @max bc
while awaiting headers
might be suspicious?
In general @Rubén Lopez Lozoya, the /dagit_info is the endpoint for our livenessProbe on dagit. The first solution I’d suggest is increasing the resource requests for the dagit pod. Are you noticing any slowness when just using dagit?
r

Rubén Lopez Lozoya

05/26/2021, 2:04 PM
It works well always. Everything is left as it was in helm chart except having 2 Dagit replicas
j

johann

05/26/2021, 2:09 PM
Hmm. Unusual but maybe you could just increase the tolerance for the livenessProbe (
dagit.livenessProbe
) in the values.yaml
r

Rubén Lopez Lozoya

05/26/2021, 2:11 PM
What param should be I increasing exactly? ^^
r

Rubén Lopez Lozoya

05/26/2021, 2:13 PM
Thanks, I will try!
m

max

05/26/2021, 3:20 PM
this doesn’t ring a bell @johann — I’m not sure what dagit_info is doing
n

Noah K

05/26/2021, 9:00 PM
I would strongly recommend y'all remove the liveness probe
1
It has no reason to be there.
j

johann

05/26/2021, 9:02 PM
Curious about this, I’ll message you directly