Independent Tool and Repository Processes and Deployment
Many of you work on internal data platforms that serve multiple teams. This release provides an oft-requested capability: deploying tools and user code independently. This was accomplished by making repositories (or a set of repositories) host-able as a gRPC server. We now have a formalized interface for both querying a repository’s metadata and invoking the pipelines within it. These can deployed separately from your Dagit server.
* Dagit can now be a fixed image which does not need to be redeployed when pipelines or solids change.
* Team-specific repositories can be deployed independent of each other, each with their own python environments.
* Repository environments no longer require the Python GraphQL stack.
We believe this is a much more stable, reliable architecture for data platform teams serving multiple internal teams.