08/06/2020, 9:41 PM
Independent Tool and Repository Processes and Deployment Many of you work on internal data platforms that serve multiple teams. This release provides an oft-requested capability: deploying tools and user code independently. This was accomplished by making repositories (or a set of repositories) host-able as a gRPC server. We now have a formalized interface for both querying a repository’s metadata and invoking the pipelines within it. These can deployed separately from your Dagit server. * Dagit can now be a fixed image which does not need to be redeployed when pipelines or solids change. * Team-specific repositories can be deployed independent of each other, each with their own python environments. * Repository environments no longer require the Python GraphQL stack. We believe this is a much more stable, reliable architecture for data platform teams serving multiple internal teams.