Hello everyone 👋 I’m new to Dagster and I’m currently working on a personal project that involves both dbt and Dagster. In this project, I’m utilizing separate Docker containers for each system, four in total. These include one for dbt and three for Dagster (the daemon, dagit, and PostgreSQL), all of which reside in a single EC2 instance. The orchestration is managed by ECS and the provisioning by Terraform. However, I’m uncertain about how these tools actually interact with each other. For example, the docker container just exists and shutsdown after dbt deps is run. As a potential solution, I’ve considered creating a small Flask server to keep the dbt Docker container running and waiting for the Dagster DAG run command to execute the models, but this doesn’t sound very efficient. If anyone has any insights or if there’s a similar project I could refer to, I would appreciate your suggestions. Thank you in advance for your time and assistance.

Hi, I suggest you to check Dagster deployment doc, well done :) https://docs.dagster.io/deployment you will find some docker deployments example and also some content around

Dagster.yml

,

workspace.yml

and

code location server

.

Hi Orest, I second @Malo PARIS’s suggestion to read through the deployment docs.

For example, the docker container just exists and shutsdown after dbt deps is run.

You mean your dbt docker container here right?