Good afternoon - I've noticed that whenever, for whatever reason, dagster-daemon dies, when I bring it up it immediately launches all missed scheduler ticks. Is there any way to prevent this behaviour from happening? Its causing extremely undesirable concurrent load issues...

Hi Dmitry - is it launching more than one tick for each schedule?

That would be great. I have multiple schedules metering out access to a shared resource. When they all launch at the same time, the resource basically dies

another feature we have that can help a lot with that is tag-based concurrency limits: https://docs.dagster.io/guides/limiting-concurrency-in-data-pipelines#configuring-run-level-concurrency if you tag each run that hits the shared resource and set a limit on that tag, they won't be able to all run at the same time

EDIT-- I checked for more recent versions of dagster and there is a bugfix for this in a newer version 1.1.13 - I will try the newest version and report back in this thread. -- @daniel sorry to resurrect this but its the only related thread I can find - My dagster-daemon is dying fairly regularly now and it's distrupting my day-to-day operations, we end up losing ~8 hours of overnight processing when this happens. I need to figure out why dagster-daemon is dying and address the root cause. Please help! Thanks, Nick

dagster.daemon - ERROR - Stopping dagster-daemon process since the following threads are no longer sending heartbeats: ['SENSOR', 'SCHEDULER']
dagster.daemon - INFO - Shutting down daemon threads...
dagster.daemon - INFO = Daemon threads shut down.

no problem - would you mind making a new post and linking back to this one once you investigate if the newer version fixes it though? that will ensure that our support process keeps track of it

Ok will do. Version 1.13 is the one I'm hoping will resolve my issue - any chance we know what the "certain conditions" are?