https://dagster.io/ logo
#dagster-plus
Title
# dagster-plus
s

Stopher

05/05/2023, 8:15 PM
Good afternoon! I have a question related to branch deployments. For some context, we are currently running Dagster Cloud Hybrid with Fargate backend. We also have two separate AWS accounts in our organization: Prod and Dev, each running the same resources and services (apart from the Fargate instance, which is only running in Prod) and each related to a branch in our repo (
main
and
dev
respectively). Until recently, we were testing our jobs during CI/CD by spinning up a local Dagster instance using docker-compose instead of using our AWS Dev account. However, we want to transition to a setup where all testing (including Dagster jobs) is performed using the Dev account, which would involve a different Fargate instance on the Dev account. Here is my question: It is possible to have our main Dagster Cloud deployment running on our AWS Prod account's Fargate instance, but have a branch deployment (i.e.: the
dev
branch) running on our AWS Dev account's Fargate instance? In other words, is it possible to have the same Dagster Cloud deployment run on two Fargate instances (one running the "main" deployment, the other running the branch deployments)?
d

daniel

05/05/2023, 8:17 PM
Hi, this is definitely possible, yeah - you can configure one agent to serve the main deployment, and another agent to serve the branch deployments, which can be set via settings in the cloudformation templates.
s

Stopher

05/05/2023, 8:19 PM
Oh awesome good to know! Does that work even if the two Fargate instances/agents are running in separate AWS accounts? Also, where can I find the CFT that Dagster Cloud uses to create the Fargate agent?
Separate accounts is no problem, although they may both need to have access to the ECR repository where your images are being stored
s

Stopher

05/05/2023, 8:21 PM
Would it work if we have two ECRs, one for prod images and one for dev images?
d

daniel

05/05/2023, 8:23 PM
Might require more work in the CI/cd to split them out, but potentially
Usually they share a single file that says what registry to look at
s

Stopher

05/05/2023, 8:24 PM
Got it, thank you very much! I'll give it a try and let you know if I have more questions!
8 Views