So my team is currently looking into Airflow, but to me it seems like we can get going faster with Dagster (seems more intuitive) and developer happiness will be better. Do any of you wish you started in Dagster or have ran into any frustrations so far? I think some of my teammates are a bit nervous about the version being 0.x, so I’m curious to hear about your experience.

went from Airflow to Dagster and never looked back

Interesting! Thanks @dwall Yeah I set it up locally and played with it a little, and seems promising. I am guessing it saved you and your team a lot of time also?

Definitely

@dwall @Makoto I am on the same journey right now, we are looking at adopting either Airflow + Dagster or Dagster alone. How are large are your data orgs / # of projects? We currently use Stonebranch for company wide workflow orchestration that is desperately needs replacing and a lot of workflows consist of tasks that call/interact with other external services - Airflow seems of a better fit for that. Dagster being 0.x is big a red flag and Airflow having Apache backing is a certainly a big plus no matter the developer experience. I am curious to learn about your journey and you’ve weighted these concerns.

@danil Hi Danil 👋 It is hard to put a number currently, but we expect our workflow to be more complex over time. I like the testability piece for sure. My teamates are comparing with Airflow, but sounds like it is not easy to setup and not as intuitive.

@danil Have you look at Prefect? I'm like Dagster's testability and mocking possibility. Other considerations some public web API(to build our own dashboard, console), flow/tasks streaming, event handling . I'm currently deciding to pick between Dagster vs. Prefect

@bklau-zap I saw it! I am looking at open-source first projects, Prefect seems like commerical first and open source second.

@danil We look at Prefect briefly but decided to look at Dagster seriously this week. Cursorily, Dagster have better docs and design vision.