hi! Dagster newbie here 🙂 been looking into Dagster and very promising so far. most of the examples are towards data engineering and processing. any resource or case studies where Dagster is being used for long running, staged tasks? basically for task orchestration

Hey Rejo. Not an expert on task orchestration, but some thoughts: • You could use a dagster sensor to manage these tasks. I.e. check for existence of new tasks and kick off runs for these tasks, or to cancel runs if tasks have run for an excessively long time • Dagster can execute tasks concurrently via the multiprocess executor, and you can set limits on the max # of concurrent tasks • Dagster handles task / run failures and has retry policies for these cases Happy to take on any questions about specific dagster capabilities, or raise them to the team