Hello all, I am noticing a strange issue in my dag...
# ask-community
a
Hello all, I am noticing a strange issue in my dagster k8s deployment using helm. Every day for the past week the prod (only) environment reports daemons not sending heartbeat at 07:30ish. I tried to upgrade and downgrade version but none of those worked. I have the same configuration in non prod and prod environment. Has someone else experienced similar issue? Is there a way to somehow get failure logs out of those daemon threads? Thanks in advance! Left picture: prod env Right picture: dev env
🎉 1
a
Interesting... do you know if there is like a procedure that runs everytime at 07:30 in prod? Maybe your devops team knows, but you can also go to the pods of your cluster and get the logs. Are you familiar with Kubernetes? If so just go to the namespace you installed dagster and check the pod logs
a
I have checked, i don't see anything triggering specifically at that time. And as far as the cluster memory concerned it seems to be ok and not peaking at any point.