https://dagster.io/ logo
#ask-ai
Title
# ask-ai
p

Pierre Cadman

03/05/2024, 4:53 PM
Can different asset partitions be computed by different functions
s

Scout

03/05/2024, 4:53 PM
Yes, different asset partitions can be computed by different functions. In Dagster, you can define assets that are partitioned by multiple dimensions using a
MultiPartitionsDefinition
. This allows you to create assets that have partitions for each unique combination of dimension partitions. Here's an example from the documentation that illustrates how you can define an asset with multi-dimensional partitioning:
Copy code
python from dagster import (     AssetExecutionContext,     DailyPartitionsDefinition,     MultiPartitionsDefinition,     StaticPartitionsDefinition,     asset, )  @asset(     partitions_def=MultiPartitionsDefinition(         {             "date": DailyPartitionsDefinition(start_date="2022-01-01"),             "color": StaticPartitionsDefinition(["red", "yellow", "blue"]),         }     ) ) def multi_partitions_asset(context: AssetExecutionContext):     if isinstance(context.partition_key, MultiPartitionKey):         <http://context.log.info|context.log.info>(context.partition_key.keys_by_dimension)
In this example, the asset
multi_partitions_asset
would contain a partition for each combination of color and date, such as
red|2022-01-01
,
yellow|2022-01-01
,
blue|2022-01-01
,
red|2022-01-02
, and so on. If you want to compute different partitions using different functions, you can define multiple assets, each with its own function and partitioning scheme. You can also use conditional logic within a single asset computation function to handle different partitions differently based on the partition key. It's important to note that while you can define multiple assets with different partitioning schemes and computation functions, each asset computation function should be responsible for computing the partitions relevant to that asset. If you have a more complex use case where you need to compute different partitions of the same asset using different functions, you might need to structure your assets and dependencies in a way that accommodates this requirement.
2 Views