https://dagster.io/ logo
#ask-ai
Title
# ask-ai
t

Tara Larrue

02/13/2024, 2:33 PM
Can I create a graph_multi_asset with partitions using dagster 1.6.3, where not all partitions have to be materialized based on conditions within the asset definition?
s

Scout

02/13/2024, 2:33 PM
As of the information available in the documentation, when using
@graph_multi_asset
to create a graph-backed multi-asset, by default, every asset produced by the graph must be materialized. This means that attempting to selectively execute a subset of assets defined in the graph-backed asset will result in an error. However, if the underlying computation is sufficiently flexible, a graph-backed asset can be subsetted to selectively output a subset of assets. This is done by running each op that is a dependency of the outputted asset. For example, if you wanted to selectively materialize
foo_asset
, Dagster would run
foo
and
bar
. If you wanted to selectively materialize
baz_asset
, Dagster would run all three ops (
foo
,
bar
, and
baz
). The documentation does not explicitly state the ability to conditionally materialize partitions within a
graph_multi_asset
based on conditions within the asset definition. It does mention that you can create partitions for assets and materialize individual partitions, but it does not provide details on conditional materialization of partitions within a
graph_multi_asset
. If you need to materialize partitions based on certain conditions, you might need to implement custom logic within your ops to determine whether to produce an output for a given partition. Alternatively, you could use dynamic partitioning strategies if your partitions are not known ahead of time and need to be determined at runtime. For more specific guidance on implementing conditional materialization of partitions within a
graph_multi_asset
, you may need to refer to the latest documentation or seek further assistance from the Dagster community, as the provided documentation does not cover this exact scenario.
2 Views