Hello! Is it possible to set slack notification on a pipeline run error? I was having trouble finding this in the docs? Also I know Airflow and Dask are options for execution, but couldn’t Cloudwatch event triggering an ECS task work just as well?

👋 hi Braun! Right now we don’t have an explicit hook to ping Slack on pipeline run failure, but we do have a Slack integration from within solids: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-slack so you can at least notify in Slack on solid execution errors. Re: ECS, our plan is to support a wide range of execution substrates over time, but we haven’t yet built an ECS integration. We’ll keep this in mind!

I think solid execution errors is probably better anyways. Is it odd though to just run the whole pipeline inside a single container? I think I am a little less concerned with a container per solid, say like the Airflow kubernetes operator. Rather just download a dagster workflow into a container and then just run the cli or python script. This is what I currently do for dbt. Thinking of dagster as a way to mange the running of dbt with ability to extend it for running things pre and post the ‘dbt’ solid.

re: one container, that should be fine, especially if you’re delegating the work to other systems (presumably your dbt is invoking workloads on snowflake/redshift/etc.) FWIW, we’re big fans of dbt, and we plan to build a first-class integration with dbt. No concrete details on timing yet, but I’ll keep you posted!