How can I export a PNG/SVG of my Dagster pipeline ...
# random
a
How can I export a PNG/SVG of my Dagster pipeline programmatically?
s
ask your question on the #ask-ai channel, you are more likely to get an answer there 🙂
👍 1
s
I'm looking into this as well; my case is graphs of SW-defined assets; looks like I can access a representation of the graph once it's been loaded into a Definitions() class via defn.get_asset_graph(), which returns a nested dict of upstream and downstream edges by asset key. So probably a quick few steps after that to load into something like networkx, label based on depth, and then plot as a multipartite layout. I'll try and circle back later this week or next if I have time.
a
Great, @Stephen Snyder thanks for the message. Look forward to hearing from you.
s
something like this:
Copy code
from dagster import (Definitions, 
                     define_asset_job, 
                     load_assets_from_modules
                        )
import networkx as nx

'''
in my case I am loading SW-defined assets from modules
'''

defs = Definitions(
            #your SW-defined assets+job here
        )

graph = defs.get_asset_graph()

G = nx.DiGraph()

for node,links in graph.asset_dep_graph['downstream'].items():
    G.add_node(node)
    G.add_edges_from([(node,link) for link in links])

# find root nodes with no upstream parents
root_nodes = [node for node in G.nodes() if G.in_degree(node) == 0]

for rt in root_nodes:
    G.nodes[rt]['depth'] = 1
    descendants = nx.descendants(G,rt)
    for desc in descendants:
        G.nodes[desc]['depth'] = nx.shortest_path_length(G,rt,desc) + 1

pos = nx.multipartite_layout(G,subset_key='depth')
nx.draw(G,pos=pos, with_labels=True, font_weight='bold')
then you can decide if you want to use matplotlib and figure save options to push it out to PNG, or if you want to make your svg directly from the networkx graph + position layout using something like pyCairo
a
Thanks a lot. Is very helpful
🎉 1