Update (07/03): We just pushed out 0.5.0! Feel free to poke around. We’re very excited about this new core. We are a US-based team so we are taking a few days off, but feel free to play around. You’ll be hearing a lot more from us next week. 🙏🏻🙏🏻🙏🏻 Hey everyone, we wanted to give you an update on the current state of the project as of 6/6/19. Right we are on version

0.4.3.post4

. We are working towards a new release,

0.5.0

, planned for July 3rd. This release will include many new features, bug fixes, and improvements. Given that the release will include some breaking changes in the API as well as some fundamental new concepts, we want to be fully transparent so that there are no surprises. Here is a (not exhaustive) list of changes that will be released: • Substantial improvements to the documentation, including topic-based guides, such as testability, integration with Airflow, Dagster for data science, and others. • A new fundamental unit of composition, similar to Airflow subdags, but far more first-class. Solids can themselves be graphs of solids, which can then compose/recurse to arbitrary subgraphs, along with Dagit support for navigating that. • A new more literate python DSL for building a dependency graph, as a layer above the dictionary-based approach currently provided. • Optional Python 3 type annotations for building the signatures of solids, to reduce verbosity. • Support for execution on Dask and local multiprocessing in addition to local single process and Airflow. • Fan-in dependencies for consolidating that allows a single input to depend on many outputs. • [breaking] We are replacing the notion of context definitions with mode definitions, which we believe will make this part of the system far more approachable and opt-in. • breaking] Renaming some definition inputs for greater consistency and understandability. We are really excited for these new concepts, and we believe they will provide a stable, compelling base of features for the medium term. This will still be an evolving system, but we will be very deliberate and strive for backwards compatibility whenever possible. Thanks so much for using and experimenting with Dagster!

@schrockn where are y’all moving to??

Oh just within San Francisco

hey @schrockn - I’m cruising through the 0.5.1 documentation right now (really great stuff, btw) and noticed that you previously mentioned there would be a guide around integration with Airflow. Does that exist somewhere?

@dwall good catch, not yet!