Trying to migrate to Dagster from Prefect, and we are trying to figure out a good workstream for us. We deployed with helm charts, and currently have the user code separated from the dagit&daemon and manage the workspace definition on a k8s configmap. This is cool since we don’t have to upgrade all of dagster everytime we change dagster code, and I like the decoupling. I think the problem for me is trying to wrap my head around how to integrate it with the CI so that multiple people trying to deploy user code changes don’t overlap with each other. I ideally want the CI setup so that if we have 2 pull requests open to merge changes to the user code, both changes are reflected in a dev deployment of dagster. Can’t seem to think of any good way to do this with helm charts.. especially since I can’t template values. (For Prefect, user code equivalents wern’t their own grpc servers, so they were just compounded on the CI without tearing anything down) Maybe this isn’t a good development workflow 🤔 curious to learn if there’s a better way for concurrent dagster code development?

Hi Jaewoo - it's true that right now even if you have the separate user code helm chart deployed, that it assumes that there is a single set of code deployed, so typically the CI/CD would be triggered on merge rather than on a PR. We've discussed supporting more of a service discovery model where individual code locations can pop up and get recognized by Dagit and I could imagine that working well for a workflow like this - that improvement is being tracked here but doesn't exist yet: https://github.com/dagster-io/dagster/issues/6295 I think a common workflow right now is that development happens locally using

dagster dev

, then PRs are merged into a shared dev/staging deployment, then later deployed into a prod deployment. Finally, with zero intention to upsell from open source dagster, just as an FYI - the workflow you're describing where open PRs can co-exist in isolated deployments before they're merged sounds a lot like the use case that motivated branch deployments, but at the moment that feature is only available in the cloud product: https://docs.dagster.io/dagster-cloud/developing-testing/branch-deployments

Thanks for the reply! Yeah it seems like branch deployments are exactly the setup we are used to atm haha. We’ll have to evaluate open source vs cloud perhaps. Thanks!