https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Abhishek Agrawal

02/13/2023, 2:11 PM
Hi all, had a question around dagster deployment in OSS - what's the preferred way of maintain dev, stage and prod versions of dagster pipelines? Could someone share any resources for the same?
@sandy @owen
o

owen

02/13/2023, 10:34 PM
hi! Here's a guide covering this topic, but the gist of it is that you can use an environment variable to programmatically distinguish what environment you're in, then provide different resource configuration per-environment.
a

Abhishek Agrawal

02/14/2023, 1:04 AM
Thanks @owen. I am still not sure how to use this to point to separate dagster instances. Can that be done via resource configuration too?
o

owen

02/14/2023, 1:06 AM
ah I see -- if you want fully separate dagster instances for these environments, then you'll need to have separate deployments for dev / stage / prod
i.e. however you have deployed your production dagster, you'd want to copy that for your other envs.
a

Abhishek Agrawal

02/14/2023, 1:11 AM
what's the preferred approach here actually? in the approach you explained above, we have 1 dagster instance, and pipelines from all environments would sit there? Do people not have fully separate instances for dagster?
If it's easier with Dagster Cloud, we would be happy to move to that but our requirement is to have separate instances of Dagster. Could you assist us with that? More than happy to get on a call if that's easier.
@sandy would we be able to manage separate dagster instances with Dagster Cloud?
s

sandy

02/15/2023, 9:59 PM
It is typical to have different instances for dev / stage / prod environments. Dagster Cloud does enable managing separate Dagster instances (it calls them deployments). @Travis McKinney can help out with questions about this
a

Abhishek Agrawal

02/16/2023, 8:30 AM
Thanks @sandy! Hey, @Travis McKinney, I have signed up for Dagster Cloud. I am going through the Kuberneters path but I'm not a DevOps dude so it's getting a bit tricky. Could I get some help?
My manager is keen for Dagster Cloud, we just need to implement it.
I have scheduled to meet with Tim Castillo on Tuesday. My questions are technical in nature as am struggling a bit to get started with adding code locations.. there's not much of DevOps support for me so I'm on my own. Would be great to get some help there. @sandy @Tim Castillo @Travis McKinney
t

Travis McKinney

02/17/2023, 4:23 PM
Hey @Abhishek Agrawal, look forward to chatting
a

Abhishek Agrawal

02/20/2023, 9:35 AM
Me too, thanks!