Guys, I wanted to have one or more gRPC servers that provide repositories. I understand I need to make the workspace.yaml looks like the one below:
My question is: how do I dynamically add new gRPC servers without rebooting Dagster instance?
Is there a lazy loading function that could return a list of gRPC servers?
03/06/2021, 3:41 PM
Hi Xu - not currently but this is on our immediate roadmap. We should likely be able to support this in dagit as part of the upcoming 0.11.0 release in just a few weeks. (So you would be able to modify workspace.yaml and then be able to reload the workspace from within dagit without restarting it)
03/06/2021, 4:03 PM
ack ty daniel
a follow up question: is that possible to construct pipelines using solids from different repos from different gRPC server?
03/07/2021, 12:10 PM
Kind of, watch my preso for the solution I've been using, basically the pipeline is built using blocking proxy objects with the actual logic in another daemon
03/07/2021, 7:12 PM
@Noah K do you have the link to you preso?
@daniel regarding the followup question ^^
03/08/2021, 8:00 PM
That's something we'd like to enable in the future with some changes to the way the orchestrator process works, but isn't currently possible.