I wanted to understand if there’s an example where assets and ops interplay? For example, a collection of ops is what leads to the creation of the asset. Some pipeline I’m importing is using assets, and another is using ops. Whats the best way to combine them? Also, with AssetGroups, how do i get it to turn into a graph instead of a job? Thanks!

hi @Charles Leung! For the first part of the question, today's release will contain preliminary support for "graph backed assets", which allow you define a graph of ops which work to produce a single (or multiple) assets. This looks something like

from dagster import graph, AssetsDefinition

@graph
def my_graph(foo, bar):
    ... # normal graph stuff
    return asset_producing_op(...)

my_graph_backed_asset = AssetsDefintion.from_graph(my_graph)

At least for me, the main thing it gets me is the ability to reuse ops across assets

Hey owen! thanks for the information! Our current pattern is designing an end to end graph that composes of other graphs. With jobs, i’m not too sure how that would work but with a .graph i suppose i could then create a graph that generates assetgroups 🙂 . Your solution helps! Let me know if I’m doing some kind of anti-pattern however 😆

the main issue with that solution (and it's a pretty big one 😛) is that it wipes out the asset-information (as that's stored at the job level at the moment). This means that running it will not result in AssetMaterializations being generated, nor will those assets show up in dagit. I think the ideal would just be to combine everything into one big asset group if possible

Something to add here is that it's possible to add asset groups together using the

+

operator. In a sense, that's the equivalent of combining multiple graphs into a single mega-graph.