People keep saying things to this effect, I'm assuming Dagster is in the same camp as Prefect, but I have no idea what this difference in design choice is, and feel like I really should. Edit: I don't have much experience with Airflow. I'm assuming this is an architectural decision (how work is divided on k8s?) and not just ergonomics.

The difference with Airflow is that you can use

PythonOperator

to have the Airflow instance itself do computions -- that's what people are saying to avoid. Using something like a KubernetesOperator would be more similar to how Dagster works by default. On the other hand, if you deployed Dagster on an EC2 instance and just ran all your jobs on that box, that would also be problematic. I thought this was an interesting article: https://towardsdatascience.com/apache-airflow-in-2022-10-rules-to-make-it-work-b5ed130a51ad?gi=b4f0b16aac2b

Thanks a lot, had no idea.