https://dagster.io/ logo
#ask-ai
Title
# ask-ai
e

elgreco

02/28/2024, 1:18 PM
Can you have nested graph assets, where the AssetDefinitions._node_def is a graphdefinition, and in that graphdefinition, the node_defs are also graphdefinitions?_
s

Scout

02/28/2024, 1:18 PM
Based on the documentation, it is possible to have nested structures within Dagster, such as op graphs that contain other op graphs. However, the documentation does not explicitly mention the nesting of
graph_asset
definitions where the
AssetDefinitions._node_def
is a
GraphDefinition
and within that
GraphDefinition
, the
node_defs
are also
GraphDefinitions
. The closest concept mentioned in the documentation is the use of
@graph_asset
and
@graph_multi_asset
decorators for defining graph-backed assets, which allow you to specify dependencies between ops within a graph to compute an asset or a set of assets. These decorators are used to create assets that are computed using a graph of ops, and the dependencies between the ops are specified inside the body of the decorated function. For more complex scenarios involving nested graphs, the documentation suggests referring to patterns such as re-using ops, handling multiple inputs, conditional branching, fixed fan-in, and using dynamic outputs within op graphs. However, it does not provide a direct example or explicit support for nested
graph_asset
definitions as described in your question. If you are looking to implement such a nested structure, you may need to experiment with the available decorators and functions to see if they can be composed in the way you are envisioning. It's also possible that this kind of functionality could be subject to limitations or require specific patterns to work correctly within Dagster's execution model. For the most accurate and up-to-date information on this topic, it would be best to consult the latest Dagster documentation or reach out to the Dagster community for guidance on implementing nested graph assets.
5 Views