Hi everyone! I'm an ML Engineer working to build and ship recommender systems using Dagster. Love the tool, great job devs! While learning Dagster, I found myself hunting for a start-to-finish working example - a Minimum Viable Pipeline. I couldn’t find one, so I built it myself. I wrote it up in a blog post here, or you can directly clone and run the Github repo. This simple example just processes a string (select the input, change case, strip punctuation), which I hope can be an abstraction for all kinds of complex Data/MLOps. It's just 120 lines of code, and explains all the Dagster basics you need to get something running. Thanks for reading!

Nicely done, @Aabir Abubaker Kar! I've added this post to the community spotlight on our community page. 😄

This is great! would have been very helpful as i was learning the ropes. One interesting take is this:

Be brutal with cutting out redundant concepts. For example, you don’t really need the Dagster

resource

concept at all. It’s just a nice-to-have, and I used it to configure paths and database connections. But in most cases, it’s non-essential.

i think this is good advice, especially to start with. dagster has a ton of nuance, and i found it very overwhelming to grok. for example

dagster_snowflake

is a very nice abstraction that can save a lot of code, but it also requires a lot of additional knowledge. one way im thinking of trying to train people on this faster is to come up with a sort of chipotle-like "job menu" for building things. 1. pick your resource 2. add your ops 3. throw it in a job! 4. choose your invocation path something like that. @Aabir Abubaker Kar i'll also direct folks to this to get started. thanks!

@Aabir Abubaker Kar Nice job! I love the minimalism

Hi @Aabir Abubaker Kar, it’s a great post! In addition to our community spotlight. I’ve recently created this Github Discussion to help share projects that are using Dagster. Feel free to share your post and repo there! — Figured it might be a great place to get folks’ ideas/feedback on the post too.