https://dagster.io/ logo
#ask-community
Title
# ask-community
v

Vitaly Markov

05/19/2023, 5:07 PM
I consider implementing a custom daemon with complex scheduling logic and dependency management, similar to
AssetDaemon
. In theory, it is possible to implement this logic using tons of sensors running on every tick. But I am not sure about the performance if I have a few thousand assets to manage eventually. Is it possible (from architecture standpoint) to add and maintain a custom daemon without changing Dagster core? Did anyone else try to do it? Any examples? Thank you!
Hmmm! It looks like it is possible to define one super-sensor which can start any various jobs in one repository. Feature is marked as experimental, but seems to be working. In theory, it solves my problem. No need for a separate daemon, just one sophisticated sensor is enough.
o

Odette Harary

05/22/2023, 2:38 PM
Hi Vitaly - Let us know if you have any further questions on this!