Hey all. I ended up deciding against using Dagster on my new project, and figured I'd share why in case the Dagster team finds it useful.
I'm building a new product, which is currently boostrapped, though I may try to raise funds next year. The product requires building quite a few batch data pipelines, and I wanted to build the first version in a way that avoids doing my own infra work. That basically means use a cloud product, rather than host my own thing. I also wanted something with transparent pricing that would fit my current bootstrapper's budget.
Dagster was my first choice based on reputation. Other cloud-based solutions I was aware of included Astronomer.io, Prefect.io, Temporal.io, modal.com, and Airplane.dev -- the last one not really having a reputation for data orchestration, but it's a thing I've used in the past.
In the end the deal killer for Dagster Cloud was price. I concluded that $0.04 per compute minute could get out of hand fast. E.g. if I have a dozen tasks that each run daily for a few hours, that could easily add up to 1000+ hrs/mo, which would cost $2,400 or more. As a point of comparison, The same amount of compute time using on-demand t3.xlarge EC2 instances would cost $166. Not that I was considering EC2, but it's a useful point of comparison.
In the end I decided to build some of the really early stuff on Airplane ($10/mo), and currently plan to build the bigger workloads on Temporal Cloud. Depending on what assumptions you make about memory resources in Dagster Cloud (I couldn't find this online), Temporal Cloud costs somewhere between 2% and 10% of what Dagster Cloud charges on a per GB-hour basis. At the end of the day I expect to be able to keep costs for v1 to a few hundred dollars per month using this approach.
I'm not sure if Dagster Cloud is missing out on other business for the same reason. But for me the pricing really stood out as uncompetitive compared to alternatives. Figured I'd share my experience because I like Dagster as a piece of software, and hope you guys can be successful commercially as well.
09/29/2022, 5:28 PM
Hey Nathan, thanks for the feedback. If you're available next week, I'd like to ask you some questions about your pricing feedback. I'll dm you now.