Hey folks, we are just getting started with Dagster and we are wondering if we should have two separate deployments of Dagster (dev and prod) or a single deployment where you would configure the jobs differently depending if they're running on dev or prod. Do you have any strong opinions about these approaches?

We use separate deployments to keep concerns completely separate (our dev/prod stacks run in different availability zones). We use an environment variable along with

os.environ()

when creating dagster resources to make sure we get the right credentials in each env

Thanks, @Isaac Harris-Holt! That's helpful! I guess that's will also be our setup once we get a bit more experienced with Dagster. The benefit of a single deployment for us who is that it would allow to iterate/experiment more quickly. We have a monorepo and I just think that waiting for production deployments could slow us down. Also I don't want to bug devops too much right from the outset :)

Well if you are using Docker or Kube, your actual code repos are separate, which helps with the quick iteration 🙂

yeah you could consider having dev vs. prod code repos in one deployment - one danger of having everything in one code repo on one deployment is that if someone pushes a change that breaks a job compilation the whole deployment will come crashing down and no one can launch / view jobs until it's fixed. But that's also something that good automated testing should be able to catch.

I deploy on k8s so yeah it's a dev deployment and a prod deployment

we use k8s, we deploy dagster and all our code using helm charts and skaffold on dev. with skaffold, build-push-deploy only takes seconds (provided that you have speedy docker registry).