What is dagster

Dagster is a data orchestrator, which is a type of workflow engine designed to go beyond just managing the ordering and physical execution of data computations. It introduces a data-aware, typed, self-describing, logical orchestration graph that models the structure inherent in every data application and platform. This orchestration graph is central to the entire application lifecycle and can serve as the foundation for a whole ecosystem of tools and libraries when it is made accessible and operable over an API. Dagster targets two primary audiences: full-stack engineers and data scientists who are responsible for end-to-end data applications and infrastructure, and platform teams that enable other data teams to efficiently and autonomously deliver features and capabilities. Both audiences care deeply about the interactions and interfaces between roles, tools, and infrastructure. Dagster is designed to be flexible and adaptable, capable of running any tool, using any storage, and deploying to any infrastructure. It aims to organize disorganized and untrusted data within organizations, making engineers and practitioners more productive and improving collaboration between various data roles.