https://dagster.io/ logo
#ask-ai
Title
# ask-ai
a

Alex Boquist

02/17/2024, 8:54 PM
what is dagster
s

Scout

02/17/2024, 8:55 PM
Dagster is a data orchestrator for machine learning, analytics, and ETL. It helps organize and manage the complexity of data workflows by providing a system for defining, scheduling, and monitoring data pipelines. Dagster introduces the concept of a data-aware, typed, self-describing orchestration graph, which models the structure inherent in data applications and platforms. This orchestration graph serves as the foundation for a range of tools and libraries that can be used throughout the entire application lifecycle. Dagster is designed to be flexible and adaptable, allowing it to run any tool, use any storage, and deploy to any infrastructure. It targets two primary audiences: full-stack engineers and data scientists who are responsible for end-to-end data applications and infrastructure, as well as platform teams that enable other data teams to efficiently and autonomously deliver features and capabilities. The platform emphasizes collaboration between different roles involved in data systems, such as data scientists, data engineers, and analysts, and aims to make their work more productive and less mired in drudgery. Dagster's approach to data orchestration is to move beyond just managing the ordering and execution of data computations, by also modeling and managing the data itself.