blob wave started evaluating dagster to replace some our current ETL orchestration process and hope to use to develop future ETLs as well

Hey everyone, I'm coming from backend development and taking my first steps in data pipelines with dagster. I'm finding it difficult to adapt to the dagster mindset coming from an OOP and procedural programming perspective. How can I bridge that gap please?

Hey everyone. I’ve been staring at the docs and following GitHub releases since 0.17 but haven’t really had a good chance to disrupt our current (and legacy) pipelines with Dagster. Now, however everything is up for grabs and since my team is frantically learning the ropes around dbt it seems like Dagster would be an awesome fit to orchestrate it all! I’m currently looking for clues to if a forever-running sensor that listens to, say a RabbitMQ queue would be a smart move to start jobs or if I’m missing the point entirely.

Hello everyone! I am leading a team of six people responsible for refactoring the backend code and data pipelines at Delfos IM to prepare our company to keep growing and scaling our technology. We are evaluating Dagster as an orchestration platform to substitute our mixed Airflow and custom solutions. We have not started working with Dagster yet, but the design principles behind it are the big selling points for us, as they align very well with what we have been experiencing. We will have an event on October 24 and 25 when the team will meet and focus on running a proof of concept of re-writing some of our ETL in Dagster. The workload is focused on timeseries extracted from multiple external sources and written to our PostgreSQL + TimescaleDB server, followed by business-driven data-transformations (we will not get ML involved in the first steps). Any advice on how to setup for this evaluation is well appreciated!

Hello, team! I'm Mavs, working on a disease surveillance and currently using dagster pipeline along with google cloud platform (GCP), python (VM notebooks, colab), github and etc. I'm not literally a data scientist or software engineer or computer scientist and tbh I'm not well versed to ETL programmatically but I wanted to learn along the way. For context, the platform and architecture was developed by an external DS team and briefly introduce the basics of the tools developed by them. I was having a hard time to open/connect to our dagster localhost:3000. I think we are using to old version of dagster since we started using it since 2020. On connecting to dagster localhost, if there are pipelines that are unsuccessful/failed, we tend to run it manually on dagster UI via localhost:3000. Steps on connecting to dagster: (1) open SDK shell terminal; (2) authenticate, (3) paste the ssh link from GCP VM instances Before, I was able to connect and open dagster localhost:3000, but now, there's seem a problem and I can't find a way to solve/debug it. I noticed in the terminal some warnings (image 2) and don't even know if that's related to my problem. Is/are there a way for me to debug or connect to the dagster UI/localhost:3000? (image 1) I've recently heard of the dasgter cloud, if we had a existing repo, can we connect it to the dagster cloud even if the dagster version was more early than the recent ones? Also, is it possible to get a demo of dagster and dagster cloud somehow? Thank you!

Hey All, I’m Nathan, I’m interested in working on Dagster workflows. Been using ssis and python for a long time, tried out prefect but after all the big changes in 2.0 wanted to give this a try and see which one would be better to use where I work.

Hey Dagster people wave anim I'm an ex-Airbyter, now leading the data platform at Unmind. Looking to have a play around with Dagster to see if it can take-over Airflow as my go-to orchestrator 😄

Hey All, I am looking to play around Dagster and could see our next stop other than Airflow in our DataPlatform. 🙂

Hey! 👋 Thought I'd join the Slack after mucking around with Dagster the past couple days for a personal data pipeline using Cloudflare R2 and DuckDB. Starting to get my head around the concepts now and appreciating the full power of Dagster and the Cloud offering. Excited to keep learning. My background is varied but I mostly create Web-based visualisations and my day job is working on exactly that at Octopus Energy (think renewables, consumer energy, etc).

Hi everyone! I'm a data engineer currently building a data platform and have decided to use Dagster as our orchestrator 🙂. Currently deploying it on Kubernetes (EKS)

Hi everyone! I’m a wannabe Data Engineer, currently a Backend/Ops developer. I’ve come to slack to ask some q.s about pipelines. We built one at the company I’m with but it’s constantly falling over because it’s running on Lambda 😬 Hoping this place will give me some insight into real Data Engineering!

Hi Everyone, I'm currently the Head of R&D for Central Group. We are looking Dagster as a potential replacement/augmentation to airflow for our Data Pipelines.

📣 Hello Dagster enthusiasts. We are looking to expand our use of Dagster! 💻 I am Head of Data Scientist at Preteckt. Our Startup designs AI to support vehicle fleets 🚚 🚌 by analyzing data to predict maintenance issues. My favorite aspects of being part of a startup are seeing the growth in our product and vision, and mentoring our Junior Data Scientists. 🏫 I was introduced to Dagster through a CoRise course in I took in August. D I implemented one of our development products in Dagster and am really impressed with its capacity to scale. Based on that experience, our team will be moving our core products to the Dagster framework. If you have MLops experience in Dagster, I would like to hear from you (please DM me). We are considering (a) working with a Dagster consultant who would help us set up and efficiently use the Dagster framework, or (b) hiring an MLops engineer with Dagster experience. If you think you would be a good fit in either of those roles, let me know and we can start a conversation. Our company culture values honestly, learning, and work/life balance. 🔗 Github, LinkedIn

Hello Guys, we send you some greetings from Hannover, Germany ! We are data engineers and architects working at a big insurance company. We are on an exciting journey setting up our modern data stack based on aws services and snowflake cloud database and dbt. Now we are starting with a proof of concept with dagster cloud hybrid. Maybe that's our new orchestration solution for our dbt models 😎 some special greets goes out to @daniel @Sean Lopp @Shaun McAvinney - Deprecated @Christian Pudenz Max

Hello everyone, i am a data engineer located in Germany. Currently testing out the tool

Hi guys 👋, I just joined your slack-ws from Hannover, Germany 🇩🇪 I am a colleague of Max and we are currently evaluating Dagster Hybrid Cloud to orchestrate our data pipelines on Snowflake with usage of DBT. I am looking forward to get in touch with you regarding support and exchange regarding dagster+dbt. BR Christian

What’s up with Dagster taking over Germany?! Glad to see it.

Yeaaaaah this whole FiveTran thing? Doesn't work for me thank you. I'll take Dagster. Oh btw I'm datadave I'm over at https://bananas.bio

Good morning/evening/[insert your time here]! My name is Casper, and i'm a software engineer turned data engineer. I am testing out dagster dagster spin from Denmark 🇩🇰 for a customer, and enjoying it quite a lot thus far. Hoping that it can move us in a code-driven direction!

Hi everyone, greetings from OpenLineage, an open standard for metadata and lineage collection designed to instrument jobs as they are running. We’re looking for new maintainers of our Dagster integration. DM me or introduce yourself in our Slack if that sounds interesting to you!

Hello! my name is Umar I'm a data engineer at a fintech based in the UK called Currency Cloud (part of the Visa group) we are exploring new open source orchestration tool and really like the look of Dagster (only looking at self deployed solution within our VPC for security reasons) would love to speak to you about it more in detail if time permits

Hi everyone! I'm José, a data scientist at a non-profit in Puerto Rico working with public policy and data that drives it. I'm going to try out Dagster to see if it can enable our team to maintain the data's reusability and observability in the long term as our analyses expand.

Hi All, I am BI developer from Hungary and I recently started to study dagster and I have lot of question about it. I hope that I will be able to write complex dagster code as well. It is a little bit confusing for me the dagster wrapping things and I can not see the underlying data.

Hello all! I work at an AI research lab called Ought, where we're considering switching to Dagster for our data pipeline.

Hello all! I currently work in Applaudo as a AWS DevOps Engineer, we're exploring among various solutions for ML pipelines on the cloud.

Hi all!!! Data scientist, looking forward to migrate data pipelines to dagster. You are doing an amazing job for the data community! :)

Hi! I'm Director of Product Engineering at coiled.io (we manage dask deployments for our users). I started learning about Dagster when we noticed from talking to our users that a number of them were using Dagster also. Also enjoyed this article (https://dagster.io/blog/dagster-crash-course-oct-2022) today with my morning coffee. Dagster's "asset" abstraction seems pretty cool and helps me compare with some of my bad old days of running data science pipelines as makefiles.

Hey, I’m a data engineer w/ https://www.deck.tools/ - exploring Dagster for our data work

Hello All, I am @Pranit Sherkar and run a company called warehows.io We work on modern data stack and helping client to implement low cost ETL solutions using open source technologies like Airbyte Airflow along with DBT Cloud Implementation. We are also official partners of companies like DBT ,Fivetran and Hevo with cloud expertise in AWS/ GCP and Azure Check out how we approach Analytics projects in below blog https://warehows.io/how-do-we-implement-analytics-projects/ Reach out for any question or query 🙂 I am checking out Dagster currently and how we can implement this in our future low cost tech implementations

Howdy, I am @Brian Clark and I work for a university research group. I am interested in Dagster for various reasons, and joined here specifically to try to find the simplest way possible to setup a demo for my group using a single Linux server. Any pointers are welcome 🙏