How would you compare dagster to Apache Beam. Here an example of wordcount. I seems they have similar concepts, but less structured (no solid approach, so I guess more code duplications?), a bit more direct. One other difference they support streams: https://beam.apache.org/get-started/wordcount-example/ What are your comments on that?

Isn't this an apples and pears comparison? In that, Beam is like Spark where you are writing a job rather than a pipeline (of jobs).

sorry, yes you are totally right. I just found this helpful article, where it compares to airflow which can be translated to dagster more better. https://stackoverflow.com/questions/50249759/apache-airflow-or-apache-beam-for-data-processing-and-job-scheduling

np