Hello, I'm a senior quant for a derivatives market maker, and I've been using dagster as my data orchestration pipeline for my personal football stats project, www.mclachbot.com

Hello, I am Abhishek aka ABC & gonna explore dagster for using for ml job orchestration. I am an airflow user and on Kubernetes but my data science team is finding moving the jobs from yaml to airflow is not straightforward and they want more native ml orchestration platform

Hi 👋 I'm Thibault, we use dagster to orchestrate different data flows. One person in my team suggested that we use dagster so we went with it. We use it for training ML models, running ML models, some sort of ETL, ... So far, we've been very happy with it. Cheers and thanks for the tool and the community!

Hi everyone 👋, I am Sophie. We are considering Dagster to orchestrate and monitor our data pipelines. Looking forward to using it and contributing to the community by pushing the limits of the tool, causing some bugs / edge cases, mainly 😄 . Great job everyone ! 💪

Hi all, I’m Matt 👋 I manage a data team that is building out a new data lake and warehouse. We are currently using AWS Glue for all our ETL from an S3 lake to an RDS operational data store to a Redshift warehouse. We use terraform to provision all the Glue resources - this works but is getting unwieldy with the number of workflow nodes we have (and we expect to need a lot more Glue resources as we move forwards). I’m interested to look at Dagster as it seems to offer a more lightweight way to manage workflows. Anyone have any resources on Glue vs Dagster?

Hello Everyone, I'm Prasanna, We use Jenkins today to orchestrate ETL Jobs. I am currently exploring data orchestration tools to switch to data flow. Our ETL Jobs(~30 odd) are written in Scala/Spark. I am not sure if there is native support for Scala looking forward to the exploration 😀.

Hello all, I'm Xavier 👋 ! I am Data Engineer in a french corp and we are exploring Dagster to maybe use it on a new data project. Heppy to join the community 🙂

Hello all 👋. We have an existing modular Python data processing pipeline deployed as a JSON API (AWS Application/Lambda). It is mostly digital signal processing (video data) plus some ML . We recently ported the dev env to Dagster and are looking at options for production deployment. Nick

Hi Dagster team, I am new to Dagster and workflow orchestrating in general. Hoping to learn from you guys.

Re-joining the channel from my personal email (changed jobs so lost the previous account). I currently work at VMware. In my previous job, I brought Dagster as the data orchestration framework and it worked (working) out pretty well 🙂

Hi, I’m the Archival Operations Manager for the Buddhist Digital Resource Center (bdrc.io) My team handles preservation and distribution. Our code is all list based (a human prepares a list of resources in a text file, and feeds it to a bash or python script.) I’m looking to define components and reorganize them into different workflows. More importantly, I’m looking to manage and log workflow instances (so I can tell what percentage of the 10000 or so items I have in a specific backlog left to do.)

Hi I'm from a trading firm and use Python heavily for various data/research pipelines. Recently I've started to migrate from Argo/dask to Dagster.

Hi all, I'm going to use dagster to create data workflows and enrichment for an analytics tool I'm building

Hey everyone, if my name and pic look familiar, it’s because it is! I just came back from a short vacation and switched jobs into Mushlabs, a BioTech company based in Hamburg producing the next generation of food with the roots of mushrooms. Excited to build a data practice here, of course with Dagster as the star of the show :)

Hi... Here trying to learn dagster and best practices for organizing the project

Hey there - I’m a new user, and also hoping to contribute to the project as well. Excited to join the community here!

Hi there! Just started a new job, and am evaluating workflow orchestration options. Top candidates for us are Airflow, Dagster, and Prefect. Recently watched Dennis Hume's talk "Dagster: Modern Data Platform Orchestration," and figured I'd join this community to learn more about Dagster as we make our decision the next month!

Hi there! Also evaluating different options such as Airflow and Dagster. Congrats on the upcoming 1.0 😎

Hello everyone, i'm currently learning dagster for new project. Hoping to learn from all of you 😁

Hello everyone... I heard @schrockn talking about Dagster on the Data Engineering podcast and I really liked the project.

Hello all! Just joined by recommendation of a colleague as we try to debug an issue with a pipeline I’m taking ownership of.

👋 Hi there! new user here, with years of Airflow background experience, and fell in love with Dagster recently.

👋 Hi everyone! Greetings from OpenLineage! We’re creating an open standard for lineage metadata collection and analysis. A customer of yours in our community created a Dagster integration and then moved on. So we’re currently looking for new maintainers. I’m also here to learn more about the platform.

new to dagster, heard about it on the data engineering podcast. Would it be good for data transfer between two databases IBM DB2 /400 and SQL Server?

Hey everyone! Coming from Lynceus alongside @Antoine Valette, we're currently using Dagster in production to manage our ML & Data engineering jobs. I will be focused on MLOps topics and how to handle hundred of models' life cycles. Looking forward to use Dagster to the fullest!

Hello everyone, I am new on Dagster and have some simple questions. I've created a JOB but it launches whenever a run

dagit

. Is there a way of starting the Dagit UI without launching the jobs ? I want it to list the jobs on the UI but I want to trigger them myself. Thank you so much!

Hii everyone, nice to be here 🙂

Just joined today after being frustrated with airflow. Hoping dagster can be a less painful solution for my use-case

Hey guys, I just joined your Channel because we are currently evaluating orchestration tools with our data team and want to take a closer look on dagster 😀 Is it possible to get a demo of dagster and dagster cloud somehow?