👋 Hey <!channel>!
Dagster 0.6.0 -- Impossible Princess 👸
We just pushed up a major new release of Dagster code-named Impossible Princess. We’ve been heads down working on this for the last few months, incorporating feedback from folks in this channel.
The feedback was pretty consistent: Dagit is a really high quality frontend, the programming model has a lot of promise, and the focus on local development and testing is great. However, it was also very clear that you wanted to use Dagit as a monitoring and debugging tool for production pipelines, and you wanted an out-of-box, cloud-friendly scheduling and execution.
That’s exactly what we built for 0.6.0.
There’s lots of new stuff but the big-ticket items are:
- A host-able version of Dagit, usable for production workflows. Before Dagit was a local development tool; now the same tool can also be used for monitoring and operating production workloads. You can view live-updating events, logs, and summary information from your production runs.
- A native scheduler. You can define schedules using a Python API, deploy them server-side, and then have them execute Dagster jobs. This is a lightweight wrapper over cron, an extremely reliable and proven scheduler.
- A fully supported Dagster-native, single-node, multi-process execution engine.
- Devops tools for easily deploying Dagster on aws. We added a command line tool to the dagster-aws module for spinning up a Dagit node and all the supporting infrastructure—security group, RDS PostgreSQL instance, etc.—without having to touch the AWS console, and for deploying your code to that instance.
It is still the same pluggable platform and system that you know. You can still deploy it to any infrastructure (not just AWS), execute Dagster pipelines on other orchestrators such as Airflow and Dask, and so forth. But we also wanted to provide an incredibly easy spin up process for those looking to experiment or use it for greenfield projects.
Please try it out and give us feedback! We’ll be posting to the broader world next week with more extensive tutorials and screencasts, but in the meantime please ask us questions let us know what you think!
10/09/2019, 4:34 PM
Hi @schrockn, when can we expect the new tutorials and documentation to hit as noted above?