As more people and organizations depend on Dagster, we think it’s important to be as explicit as possible about which Dagster APIs are public and which are private. Over the next few weeks we’re going to be incrementally renaming our informally private modules (all submodules of

dagster

) so that they are formally private. Users should target all

dagster

imports to the top-level

dagster

module:

# CORRECT
from dagster import op

# INCORRECT
from dagster.core.definitions.decorators import op

Renaming starts in our most recent release (

0.14.15

) with the rename of

dagster.check

>

dagster._check

. We hope this change will not affect the vast majority of users, since our public API has long been informally defined as the top-level

dagster

module. However, for users who are using private APIs or otherwise importing from

dagster

submodules, we have made some temporary accommodations. There’s a shim in place that will prevent our module renames from breaking runtime-- imports of old module names are dynamically resolved to their new locations (so

import dagster.check

is equivalent to

import dagster._check

as far as runtime is concerned). However, all submodule imports will now trigger linter warnings-- either because you are importing a formally private module (e.g.

dagster._check

) or because you are importing from a module name invisible to static analyzers (e.g.

dagster.check

). Please reach out in #dagster-support if this transition causes you any other problem!

Do you enjoy using Dagster? Join us as a developer advocate! ✨Develop product education materials ✨Represent Dagster in the developer community ✨Engage with users ✨Influence product direction on behalf of the community As the developer advocate, you will create product education materials that will help our users to smoothly onboard to Dagster and succeed in learning about the project and best practices, adapting Dagster to their custom infrastructure, evaluate Dagster for their organization, or anything in between. You will also be one of the early members of the dev rel team and will have the opportunity to help us shape the team as it grows. See full job posting and apply here! Feel free to also ping me directly if you're interested. 🙂

Help us name the 0.15.0 release! Wanna help us name our upcoming release? Now is your chance! 🌈 In the past, we’ve sourced a list of names internally and then voted internally to come up with the name. But this time we want you to help us! In the comments of this GitHub Discussion you'll see the names that have been sourced by the Dagster team. Do us a favor and vote on your favorite by adding a ❤️.

We're excited to share that we are partnering with Dennis Hume (@dhume, Data Engineer at Dutchie) and co:rise to build a course on Data Engineering with Dagster! The course launches on August 15th –sign up HERE and read more below: A well built data platform allows for fast iteration and safe deployments. This course will teach you how to design, build, and maintain a data platform that supports a wide range of data tasks. You will start by taking a simple data workflow, common to most companies, and deconstructing it to its core components. By the end of the course, you will reimplement the pipeline using modern data frameworks, running in the cloud. We'll also have special guest lectures featuring folks from the Dagster team. Want to help us share this around? Here are the LinkedIn and Twitter posts about it. Looking forward to an awesome class! 😄

<!channel> - 🍍🌞 blob sunglasses Announcing... Dagster 0.15.0: Cool for the Summer blob sunglasses🌞🍍 In 0.15.0, software-defined assets are fully stable and ready for prime time. We now recommend using them whenever building and maintaining data assets is your goal. Here's a guide on why, when, and how to use them. We’ve made a number of other big improvements as well. Here’s what’s new: • Asset config - Software-defined assets now support configuration. • Asset organization - You can organize software defined assets into groups by providing

group_name

s. These assets will be grouped together in Dagit. • Graph-backed assets - software-defined assets can now be interleaved with ops. • *Graph &amp; job input values -*`execute_in_process` and

<http://GraphDefinition.to|GraphDefinition.to>_job

now both accept an

input_values

argument, so you can pass arbitrary Python objects to the root inputs of your graphs and jobs. • Run retries - You can now automatically re-execute runs from failure, like op-level retries, but at the job level. • Redesigned partition & backfill pages - Now they're faster and show the status of all partitions. • Redesigned navigation pane in Dagit - It now organizes jobs and asset groups by repository. • Generic Outputs - Ops that return `Output`s and `DynamicOutput`s now work well with Python type annotations. • Job metadata - You can now attach metadata to jobs and see that metadata displayed in Dagit. And more • Full release notes • Migration guide A special thanks to everyone in the community who contributed to this release: @Charles Lariviere, @fahad, @Abdó Roig, @Donny Winston, @3cham, @Frank Dekervel, @peay, @bollwyvl, @trevenrawr, @Javier162380, @jrouly, @dhume, @ascrookes, @kstennettlull, @swotai, @HAMZA310, @kbd, @LeoHuckvale, @joe-hdai, @antquinonez5, @Alex Kerney, @frcode, @dwall, @proteusiq, @kahnwong, and @iswariyam! And thanks to everyone who helped us name this release: https://github.com/dagster-io/dagster/discussions/8234

📰 The next Dagster Community Meeting is happening Tuesday, June 21 at 9AM PT. Join us on Zoom! To register for a calendar invite, fill out this form. 🎙️ Here's the agenda: • Software-defined assets update (@sandy) • Ops and graphs with SDA (@claire) • Orchestrating dbt and python using dagster (@owen) • Dagit Improvements (@josh) • Dagster Cloud Update (@schrockn) • Community Spotlight (@Mollie) • Q&A! 😄 We're looking forward to seeing you there! Share it around (twitter). 😄

New Post on Orchestrating Python and dbt with Dagster! Tweet. 🙂

Reminder that the community meeting is happening tomorrow! 😄

As @schrockn mentioned in today's Community meeting (video coming soon), we are looking for folks who are interested in talking to us about their experiences with Dagster for user testimonial videos. Reach out to me if this is something you'd be interested in. I'll also be reaching out to some of y'all about this. Thanks, y'all. daggy love

For those of you who missed today's Community Meeting, ! 🌈 Additionally, We'd like your feedback! Have the community meetings felt useful to you? Is there something in particular you'd like to see at future community meetings? Let us know!

Hello <!channel>! I hope you’re having a great Tuesday. I have some exciting news for the entire community. On Tuesday August 9th 9 AM Pacific/4 PM UTC we are hosting Dagster Day and making two huge announcements that mark a new era for the product: • Dagster 1.0: The culmination of years of work, Dagster is graduating to 1.0. Dagster’s core is now formally stable and production deployments can be built using it and maintained with confidence. • Dagster Cloud General Availability: Dagster Cloud has been in Early Access since December. This has gone extremely well, and we have lots of happy users ranging from small startups to Fortune 500 companies, running on a stable, rock-solid platform. Starting August 9th anyone can sign up and start using Dagster Cloud. You can start with a free trial, and then just swipe your credit card and go. Cloud GA will not just open current capabilities to self-serve, but also include some new revolutionary capabilities that we are excited to share with the world. Sign up for Dagster Day here. It’s an hour-long event where you can learn more about the launch, ask us questions, and meet other members of the community. In that same form you can also sign up for the waitlist! If you do so before August 9th you will get a trial with double the amount of free usage than a standard account. For news on Dagster Cloud, I encourage you to join the #dagster-cloud channel where we will preview features and announce updates to the platform. There is some amazing stuff coming down the pipeline. Thanks to the entire community and our early access customers for placing your trust in us and helping us build this product. We’re excited to enter this new era and we couldn’t have done it without you.

Dagster 1.0 is coming; help us name the release! Wanna help us name our upcoming release? Now is your chance! 🌈 In the comments of this GitHub Discussion you'll see the names that have been sourced by the Dagster team. Vote on your favorite!

<!channel> - Announcing... ☎️📞👋 Dagster 1.0: Hello 👋📞☎️ ! Data teams who want to access what’s unique about Dagster now have a fully stable foundation to build on. 1.0 doesn’t include any seismic changes - rather, it marks that we’ve put the finishing touches on Dagster’s core abstractions. @schrockn committed Dagster’s first line of code four years ago. Since then, Dagster has had 463 releases and grown from an experimental programming framework to a production-grade orchestrator with software-defined assets, schematized config, rich logging, sensors, thirty three integration libraries... and more. Over 200 people have contributed code to Dagster, and its Slack community has grown to 4,300 members. Through all this, we held off on declaring Dagster 1.0, because we believe iteration results in better software - if you stick with the first approach that occurs to you, you’ll likely end up far away from the global optimum. Over that time, we made tweaks to Dagster’s core APIs that substantially improved them. Eventually, we found we weren’t making breaking changes. Last year, we arrived at stable compute abstractions - ops, graphs, jobs, schedules, and sensors. This year, we did the same with the asset layer - software-defined assets, materializations, and asset partitions. With all the core abstractions now stable, it was time to make things official and declare 1.0. 1.0 includes some other significant changes and improvements too: • Example revamp - All Dagster examples are revamped with a consistent project layout, descriptive names, and more helpful README files. • Repository-wide default executors and loggers - if specified, these will be used for any jobs which do not explicitly set an

executor_def

or

logger_defs

. • Run status sensors can now yield

RunRequests

- allowing you to kick off a job in response to the status of another job. • Input managers - when loading an upstream asset or op output as an input, you can now set custom loading behavior using the

input_manager_key

argument to AssetIn and In. • Global asset lineage graph - back and better than before. The graph clusters together assets that are in the same group, and the query bar allows you to visualize a custom slice of your asset graph. • dagster project CLI - contains commands for bootstrapping Dagster projects from scratch or from examples, with our recommended project structure. • Legacy and deprecated API removal - many deprecated APIs have been removed. Take a look at the migration guide for details. • Local-to-prod guide - a new guide on how config, resources, and environment variables fit together to make jobs and assets portable across environments. • Library versioning - Dagster’s integration libraries are now at version 0.16 and are compatible with Dagster 1.0. We’ll mark them 1.x after we audit their APIs in the same way we’ve audited the core APIs. Resources • Full release notes • Migration guide And thank you to everyone who helped us name this release!

Hey everyone! Today’s the big day… Dagster Day! celebrate Be sure to at 9:00 AM PST for the livestream (less than one hour from now!!). 🎥 Also, if you want to help us make the most of this launch and get the word out, here are some ideas we would ❤️ to get your help on! • Nick wrote a launch post you will find here, which you could retweet here. • Stay tuned to twitter during the launch, retweet posts you're excited about, and keep the conversation going • We posted to Product Hunt here and would love to see Dagster get an upvote - if you have a minute please check it out and add a comment. • Finally, we are at 5,100 Github stars and counting! ⭐ Stars are actually incredibly helpful for publicizing and expanding the community, so please chip in and star the repo See y'all there! 🌈

<!channel> - live now! 😄

📰 The next Dagster Community Meeting is happening Tuesday, October 11th at 9AM PT. If you would like to see your name in lights, and have something to share with the community, please DM me directly here on Slack. I know there are some great conference presentations being lined up for this Fall season! This is a chance to share your hard work with the community. Otherwise look for an announcement soon on the full agenda, and please block off that time to join us live. In the meantime, you can sign up for Community Meeting reminders here.

Are you going to dbt’s Coalesce conference in New Orleans? We will be hosting a Dagster happy hour on the 19th from 5:30 to 7:30. We hope you can join us. Sign up here: https://www.eventbrite.com/e/dagster-happy-hour-tickets-425213403737

Do you know of somebody looking for an overview of Dagster? Would you like to connect with the Dagster team live? 🎉 Every other Thursday, starting on October 6th, we will be hosting a live demo/introduction with live Q&A session. This is an opportunity for people new to Dagster to get oriented and learn about the framework and key abstractions. Interested? Sign up here: https://dagster.io/dagster-demo-signup

Hello Friends, Dagster-PM here cross-posting from #dagster-feedback to solicit your help in improving Dagster. We’re going through a process of prioritizing new features, example repos, integrations, improvements to our content, etc, and it’d really help to hear from you. <https://forms.gle/xVgfJbubQpfMiBJh7%7CHere’s a <5m survey with a few questions about your team, your tech, and where you want us to focus our time>. In exchange for your input, we’ll send you some Dagster sticker swag. Woohoo.

Just a reminder that the Dagster Demo + live Q&A is taking place tomorrow @ 9:00 PST / 12:00 EST / 18:00 CET. Sign up here: https://dagster.io/dagster-demo-signup

@Pete Hunt, our Head of Engineering just shared a new Dagster crash course, for anybody looking to ramp up quickly 🚗 🏁 https://dagster.io/blog/dagster-crash-course-oct-2022

Reminder that t*he next Dagster Community Meeting* is happening tomorrow at 9AM PT. On the agenda: dagster spin Community spotlights. A Updated Dagster-Airflow to make migration even easier. powerup Asset updates: Loading asset values, multi-asset sensors, and asset reconciliation 📺 Dagit enhancements 🪄 Orchestrating Airbyte with Dagster And we will leave time for your questions at the end. You can sign up for Community Meeting reminders here.

Today’s community meeting starts in 30 minutes. Join the Zoom Meeting here: https://us02web.zoom.us/j/88939083535

🎉 Community Spotlights recap from the Community Meeting 🎉 • [Blog post] Dagster Day: Thoughts for Airflow Users by Hector Robles • [Blog post] Data News: Dagster 1.0 Launch Recap by Simon Späti at Airbyte • [Blog post] Dagstermill: passing notebook results downstream in 3 steps by Stefan Samba • [Blog post] 4 Essential Steps for Building a Simulator by Devjit Chakravarti at Doordash • [Blog post] Dabbling with Dagster - Part 1 by David Jayatillake • [Blog post] Tchau Tchau Airflow, Olá Dagster by Lucas Gabriel • [Blog post] Dagster + Airbyte + dbt: How Software-Defined Assets Change the Way We Orchestrate by Tamas Foldi • [Blog post] A Dagster Crash Course by Pete Hunt • [Video]

Why EvolutionIQ Picked Dagster For Their Data Orchestration Layer▾

by SeattleDataGuy w/ EvolutionIQ • [CoRise course] Data Engineering with Dagster by Dennis Hume heart hands Shout out to everyone who contributes to this community and thank you for the support daggy love 🗣️ Besides the Community Spotlights, we’d also love to have some community members demo your use cases or contributions. The next Community Meeting will be held in early December. If you would like to see your name in lights, or have something to share with the community, please DM me directly! If you haven’t already, please ⭐ the Dagster Github Repo - this helps us raise the visibility for the open source project.

Hey all! Dagster Product Designer here. 👋 We are testing out a couple UI updates in this week's Dagster release that bring some big performance improvements to the Workspace + new improvements to the Instance Overview. You can check out a demo of these changes and you can try them out for yourself by enabling the

New workspace pages, overview pages, and top navigation

feature flag in your user settings. We’d love for you to try these out and let us know what you think! Feel free to share any feedback in this thread or in the #dagster-feedback channel.

For all of you following dagster on Twitter, you will notice we have changed our handle: we are now simply @dagster. The account stays the same, and if you were following @dagsterio you will still be getting all our updates.

📰 Hi all, we created a channel (#faq-read-me-before-posting) that highlights the structure of our Slack workspace. For new users and existing users, it contains some links to get started with Dagster and the Slack. Also, it highlights all of our channels in one place and separates them by category (e.g. Dagster Cloud users, integration specific use-cases, or just general interest). This new channel is meant to be read-only, so please don’t post there! Let us know if you have any questions daggy love

We’ll be highlighting new channels in the upcoming weeks that are dedicated to integrations that we’re actively working on to be fully compatible with the Dagster framework out-of-the-box. In addition to library support, we’ll be providing step-by-step guides to use the key features of the integration in tandem with Dagster. One of these new channels is #dagster-airbyte, with integration development being led by our very own @ben! Here’s some of his recent efforts in making the integration software-defined asset compatible. We can generate the corresponding assets in Dagster as you manage your connections in Airbyte. It’s pure magic ✨ If you’re interested in using this integration, join the channel to receive weekly updates and discuss your use-cases 👌

alert Another channel highlight! Many of you have been familiar with orchestration, even before Dagster. For those of you who already working with an existing orchestration platform, like Airflow, we’ve been making incredible strides to make an incremental migration path. You can incrementally adopt Dagster and realize the value and potential of our programming model to manage your data assets. In #dagster-airflow, @Joe has led the development of three methods to help you on this path to incremental migration: 1. ➡️ You can orchestrate Dagster runs from Airflow by using the

DagsterCloudOperator

or

DagsterOperator

operators in your existing Airflow DAGs. 2. 🤯 You can automatically create a Dagster job from an existing Airflow DAG - we’ll infer all the dependencies for you. 3. 👀 You can take advantage of Dagster’s programming model to create robust, testable, and reusable jobs to materialize your data assets - we’ll infer the Airflow DAG for you. Join the channel for updates and to discuss your Airflow migration questions dagster spin