I would be curious to hear how people create Dagster testing environments for their developers... How current structure is simple, everything that goes to

master

goes to a dagster staging deployment and whenever a new git tag is created that tag is deployed to the production instance. What we would like to achieve is something that would allow our developers to have their pipeline deployed to a testing instance whenever they (for example) push to a branch starting with

test-

. We use Dagster on kubernetes with helm, so our user deployments are defined in the helm values.yaml file. Do you see a nice and clean way of doing it? Does anyone have a better solution?

I’m also interested in this. Presumably you could run a container alongside dagit which checks for new UCD deployments then updates the workspace.yaml configmap and restarts dagit?

Wouldn't that conflict with helm? We would have to change a configmap managed by helm so that's not going to be a very clean solution...

open to better solutions!

I also would be curious to know what happens if we define in the same workspace.yaml two GRPC repositories offering the same pipelines (e.g. same name and repo but different version)

We would have to change a configmap managed by helm so that’s not going to be a very clean solution...

A user actually is contributing a fix here: https://github.com/dagster-io/dagster/pull/7343?no-redirect=1

Interesting. Is Dagster able to hot-reload the workspace.yaml file without restarting?

Not currently.

I see! From what i can see there is a graphql query that we can use to reload the workspace dynamically. With a small sidecar container we could trigger the call whenever the configmap has changed

I know a few users have something akin to what you described, though I don’t know all the details

would you mind tagging them here or sharing their contacts?

The easiest is just a full helm install for a branch, but that does have unnecessary overhead. Deploying dev branches is also pretty standard in Dagster Cloud, where we were able to build for this from the get go. It’s a bit harder to port dynamic workspace stuff back to oss Dagster, but we’re discussing it