Hi! 👋 Is there a more in depth guide on how to deploy Dagster & Dagit to AWS than this? I’d like to evaluate Dagster for Databricks pipeline orchestration.

https://docs.dagster.io/deployment/guides/aws#example has a step-by-step guide for deploying to ECS via docker-compose that you can use as a template for your own deployment - have you taken a look at that yet?

It looks like the code example link has more info which is nice but how do I build the connective tissue for continuous deployment — getting a repo’s new Databricks Dagster pipeline code pushed to our deployed Dagit on ECS?

A couple options I can think of: - you could have CI/CD run docker-compose

build

,

push

, and

up

for you - the actual pipeline code could live in EFS (or perhaps even S3 https://github.com/s3fs-fuse/s3fs-fuse) and you could mount it as a volume. https://docs.docker.com/cloud/ecs-integration/#volumes Your CI/CD pipeline would be responsible for pushing changes to the pipeline code and then restarting the user code task. Here are some other recent discussions that you might be able to draw on for inspiration: https://dagster.slack.com/archives/C01U954MEER/p1619619726153500 https://dagster.slack.com/archives/C01U954MEER/p1624547226085700 https://dagster.slack.com/archives/C01U954MEER/p1619788572275400

Thanks @jordan :blob_highfive: — that should keep me busy for a while.

Awesome! Also, check out #dagster-ecs if you’d like - there are a couple community members who have some experiences running Dagster in ECS that sometimes post over there as well.