My daemon stopped out of nowhere. Is there a place...
# announcements
l
My daemon stopped out of nowhere. Is there a place where I can see the daemon logs? Not the pipeline logs.
d
Hi Laura - there should be logs written to stdout/stderr from the process. Hopefully there should be a clue there about what's going on. We don't currently capture the raw logs from the daemon process and surface them anywhere though. If one of the daemons raised an error that was caught though, that would be surfaced in Dagit on the Status page in the left nav under Daemon statuses
l
Can I still see it if I restarted the service?
j
Is this on k8s? If the pod crashed and restarted,
kubectl logs <pod name> --previous
may work
l
I have a solid that makes a GET request. Sometimes the request returns 500. Usually dagster is handling it well, it presents a failed step and move on. In this particular case, I don't know what happened. It just stopped while waiting for the GET request to come back.
@johann No, I'm running the daemon in tmux
d
You could pipe the output of the process to a file to be able to persist it, but dagster doesn't automatically persist it for you anywhere currently.
l
ok, thanks