Hi. I'm new to dagster. I’m loving it😍 I launched dagster components on ECS. It works well. My question here is we always include dagster projects along with same repository that docker compose resides? This is because we adapt super duper strict policies on github repo access even for colleagues. I’m finding sorta apply command in k8s cronjob, which can be called anywhere as long as you specify a cluster and namespace.

My question here is we always include dagster projects along with same repository that docker compose resides?

Not sure if I’m understanding correctly, but if you’re deploying with docker compose you’d just need the docker images produced by the docker code in the gh repository

via rpc service? What I meant is similar as this post: https://dagster.slack.com/archives/C014UDS8LAV/p1689594909074219 My concern is we eventually distribute same config files such as workspace.yaml and dagster.yaml across all repositories with the way in this example. https://github.com/dagster-io/dagster/tree/1.4.0/examples/deploy_ecs

You could set up each repository to deploy its own grpc server. They would each need a dagster.yaml config (could load this in to ECS tasks, doesn’t need to be in the git repo) and only the dagit service needs a workspace.yaml, that would only need to be updated when you add/remove a code location or change its address

So the only coupling variable is my code location in workspace.yaml? Here is what I think off the top of my head. Dagster core component repo • dagster.yaml • workspace.yaml that need to include each repo code location • Whatever ECS definition spin up dagster daemon and dagit My repo has my actual workflow code • mycode • dagster.yaml that is separated one in the core repo. • Dockerfile • Ecs task definition ( whatever IaC like CFn or terraform) • No workspace.yaml?

That sounds right to me

For sure. No one want to define same things across multiple locations.

I’m not an ecs expert, do you know if there’s a way to inject a file (dagster.yaml) in to ecs containers via the task def? Akin to kubernetes volume mounts? That’d be convenient here. From aws parameter store and secrets manager I only see injecting env vars

I’m not also an ECS expert but we might try EFS volume to share same files across many tasks