Pierre Cadman
03/05/2024, 4:53 PMScout
03/05/2024, 4:53 PMMultiPartitionsDefinition
. This allows you to create assets that have partitions for each unique combination of dimension partitions. Here's an example from the documentation that illustrates how you can define an asset with multi-dimensional partitioning: python from dagster import ( AssetExecutionContext, DailyPartitionsDefinition, MultiPartitionsDefinition, StaticPartitionsDefinition, asset, ) @asset( partitions_def=MultiPartitionsDefinition( { "date": DailyPartitionsDefinition(start_date="2022-01-01"), "color": StaticPartitionsDefinition(["red", "yellow", "blue"]), } ) ) def multi_partitions_asset(context: AssetExecutionContext): if isinstance(context.partition_key, MultiPartitionKey): <http://context.log.info|context.log.info>(context.partition_key.keys_by_dimension)
In this example, the asset multi_partitions_asset
would contain a partition for each combination of color and date, such as red|2022-01-01
, yellow|2022-01-01
, blue|2022-01-01
, red|2022-01-02
, and so on. If you want to compute different partitions using different functions, you can define multiple assets, each with its own function and partitioning scheme. You can also use conditional logic within a single asset computation function to handle different partitions differently based on the partition key. It's important to note that while you can define multiple assets with different partitioning schemes and computation functions, each asset computation function should be responsible for computing the partitions relevant to that asset. If you have a more complex use case where you need to compute different partitions of the same asset using different functions, you might need to structure your assets and dependencies in a way that accommodates this requirement.