Hey folks! We are looking forward to using Dagster in a production environment. Our Data Pipelines are user-facing, so we would like to have multiple numbers replicas for all the Dagster components, to make Dagster highly available. We scale the Dagit and all the different user code deployments, although we find out that in the Chart it is not possible to increase number of replicas of the Daemon component, it is this something expected?

Yeah the daemon is not meant to be scaled horizontally ATM, do you can run only a single replica

do you know what it is the reason behind as curiosity? I though it is a deployment in charge of spinning schedule pipelines 🤔

I think it hasn't been coded in a way that allows multiple replicas. If you create two demons they will compete to run the same pipelines

ok! Clear thanks @Andrea Giardini

Only thing i'll add is that while the daemon doesn't support replicas (for the reasons Andrea mentioned - since it's managing a centralized run queue, for example), it's set up to be able to stop at any time and pick back up where it left off (catching up on schedules/sensors/etc.), without interrupting any ongoing runs. So you can't have two daemons running at once, but it should be no problem to failover to a different daemon if the first one goes down, without significantly interrupting service other than a brief delay in new runs being launched.

Hey @daniel, I can imagine for the same reason that the user-code-deployment can only run with one replica?

That’s actually due to a different limitation that we have plans to remove. There’s some added complexity of figuring out when Job definitions have changed if you have replicas.

I see, and if we patch all the replicas when a new user code deployment it is deploy that limitation it is still there?

Currently. It's on our radar to fix