What is dagster

Dagster is a data orchestrator, which is a type of workflow engine designed to go beyond just managing the ordering and physical execution of data computations. It introduces a new primitive: a data-aware, typed, self-describing, logical orchestration graph. This graph models the structure inherent in every data application and platform, and when made accessible and operable over an API, it can form the basis of an ecosystem of tools and libraries. Dagster targets two primary audiences: full-stack engineers and data scientists responsible for end-to-end data applications and infrastructure, and platform teams that enable other data teams to efficiently and autonomously deliver features and capabilities. It is designed to be flexible and adaptable, capable of running any tool, using any storage, and deploying to any infrastructure. Dagster emphasizes the interactions and interfaces between roles, tools, and infrastructure, aiming to make data within organizations more organized and trusted, and to improve collaboration among various data roles.