What's the future of ops, in light of assets?
I'm catching up on assets. Reading the introductory blog post the idea of assets seems very compelling (we do often have questions of which part of our data warehouse is how out of date, and when updates will take place). But it's a bit unnerving that the Dagster homepage / cereal example no longer makes any mention of ops. Our current pipelines are all ops, and some of that structure, as well as the explicit linkage of data flow through the job via Python function calls, is very nice. Should we expect development to focus on assets in the future? Will ops go away and we'll need to migrate to assets?
07/07/2022, 7:02 PM
Ops are a critical abstraction that will not be going away. Software-defined assets are a powerful abstraction built on top of ops that are useful for a lot of workflows, which motivated the recent restructuring of our intro docs. However, there are plenty of workflows where software-defined assets are not a good fit and using ops is actually recommended. There’s a guide on the docs site which tries to outline which scenarios are best suited for assets and which scenarios are best suited for ops.
07/07/2022, 7:07 PM
Very helpful, thank you!
07/07/2022, 7:43 PM
good to hear ops are not becoming a second-class citizen 🙂 will admit the docs re-org scared me a little as well