https://dagster.io/ logo
#ask-community
Title
# ask-community
m

Matt L

08/07/2023, 3:18 PM
How do heartbeats work? When are they sent? I'm seeing the dagster-daemon crash with this error regularly:
Copy code
dagster.daemon - ERROR - Stopping dagster-daemon process since the following threads are no longer sending heartbeats: ['ASSET']
dagster.daemon - WARNING - Shutting down daemon threads due to Exception...
dagster.daemon - ERROR - Thread for BACKFILL did not shut down gracefully.
dagster.daemon - ERROR - Thread for ASSET did not shut down gracefully.
dagster.daemon - INFO - Daemon threads shut down.
I was running backfills with ~500 partitions. Is it possible that all of the backfills need to complete before the next heartbeat is sent?