https://dagster.io/ logo
#deployment-kubernetes
Title
# deployment-kubernetes
a

Andrea Giardini

04/21/2022, 2:01 PM
I would be curious to hear how people create Dagster testing environments for their developers... How current structure is simple, everything that goes to
master
goes to a dagster staging deployment and whenever a new git tag is created that tag is deployed to the production instance. What we would like to achieve is something that would allow our developers to have their pipeline deployed to a testing instance whenever they (for example) push to a branch starting with
test-
. We use Dagster on kubernetes with helm, so our user deployments are defined in the helm values.yaml file. Do you see a nice and clean way of doing it? Does anyone have a better solution?
a

Alex Remedios

04/21/2022, 2:03 PM
I’m also interested in this. Presumably you could run a container alongside dagit which checks for new UCD deployments then updates the workspace.yaml configmap and restarts dagit?
a

Andrea Giardini

04/21/2022, 2:05 PM
Wouldn't that conflict with helm? We would have to change a configmap managed by helm so that's not going to be a very clean solution...
a

Alex Remedios

04/21/2022, 2:07 PM
open to better solutions!
a

Andrea Giardini

04/21/2022, 2:08 PM
I also would be curious to know what happens if we define in the same workspace.yaml two GRPC repositories offering the same pipelines (e.g. same name and repo but different version)
j

johann

04/21/2022, 2:26 PM
We would have to change a configmap managed by helm so that’s not going to be a very clean solution...
A user actually is contributing a fix here: https://github.com/dagster-io/dagster/pull/7343?no-redirect=1
in the same workspace.yaml two GRPC repositories offering the same pipelines (e.g. same name and repo but different version)
This should be fine except for one known bug where filtering the runs page will still show runs from the other repos (fix planned, but it’s a bit involved)
a

Andrea Giardini

04/21/2022, 2:31 PM
Interesting. Is Dagster able to hot-reload the workspace.yaml file without restarting?
j

johann

04/21/2022, 2:32 PM
Not currently.
Sorry- it doesn’t need to restart, you do have to click ‘reload workspace’ currently
a

Andrea Giardini

04/21/2022, 3:28 PM
I see! From what i can see there is a graphql query that we can use to reload the workspace dynamically. With a small sidecar container we could trigger the call whenever the configmap has changed
@johann Do you know of anyone doing this developer-environments with k8s dagster?
j

johann

04/21/2022, 3:44 PM
I know a few users have something akin to what you described, though I don’t know all the details
a

Andrea Giardini

04/21/2022, 3:47 PM
would you mind tagging them here or sharing their contacts?
j

johann

04/21/2022, 3:47 PM
The easiest is just a full helm install for a branch, but that does have unnecessary overhead. Deploying dev branches is also pretty standard in Dagster Cloud, where we were able to build for this from the get go. It’s a bit harder to port dynamic workspace stuff back to oss Dagster, but we’re discussing it
🙏 1
I don’t know if they’re on slack but @peay has been considering how to do this using CRD’s for Dagster repo locations
2 Views