06/29/2020, 5:15 PM
0.9.0 Release Plans Hi everyone, 0.8.0 was a massive release requiring substantial restructuring of the whole system. We’ve decided to release our next major version, 0.9.0, in late July. This is a shorter timeframe than our typical major release, and it will focus on polish, bug fixes, documentation, and building new features that have been made possible by the architecture changes in 0.8.0. With the upcoming release, we wanted to share an update on our plans to give you all visibility into what we’ll be working on over the next month: • Containerization: In 0.8.0, we decoupled our tools such as Dagit from user code in a separate process. In 0.9.0, we’ll push this further by working towards supporting packaging user-defined repositories in containers. • Documentation: Completely revamping our documentation, including a collection of scenario-driven examples, new overview sections, and cleaner organization. • Kubernetes Improvements: • Based on the containerization work, separating user code from the Dagit Deployment.   • Adding support for run cancellation and retries. • Adding resource limits for run coordinators. • Threading all logs back through to Dagit. • Exploring publishing Helm chart to simplify usage. • Removing cron scheduler from the Dagit Deployment • Hooks: Adding support for triggering actions based on events like success or failure. We want to support alerting, notifying other systems like PagerDuty or Slack about solid execution results, etc. • APIs: Cleaning up rough edges in Dagster’s APIs: • Fixes for the compute log manager. • Improvements to Dagster’s subselection semantics and for backfilling solid subsets. • Refining the new workspace YAML. • Moving storage configuration from run config to the instance. • Dagit: Various UI improvements and polish to clarify information hierarchy: instance vs. repo vs. pipeline scope. Updates to longitudinal views, including showing step-level success / failure. • gRPC: Adopting gRPC for internal communication between components in Dagster deployments. After this change user environments will no longer have to include the Python GraphQL stack. • Great Expectations: Adding a new Dagster integration with Great Expectations.
💯 3
🚀 4
🎉 9
👏 9
❤️ 2
Feel free to reach out (or comment on this thread) with any questions!


06/29/2020, 5:39 PM
How about telegram support in hooks? we could help if you want, that would be super usefull for us!
👍 1

Robert Lancer

06/29/2020, 6:09 PM
Looks good, for Helm having hosted images would be great


06/30/2020, 2:18 AM
Def excited for event hooks, failure notifications are good for critical pipelines

Simon Späti

06/30/2020, 7:12 AM
Thanks nate for letting us know! I think almost all of the features will be beneficial for us. Especially exited about the extended Great Expectations integration, I believe this is very crucial to pipelines success story. That you can guarantee the DQ in the long run.


06/30/2020, 9:00 AM
Thx for the update - Regarding Dagit - multiple selection would be beneficial to triage the runs.