https://dagster.io/ logo
#ask-ai
Title
# ask-ai
e

elgreco

03/06/2024, 2:59 PM
Can I make a monthly partitions definition return a the partition keys in a DBIOManager instead of getting a partition window?
s

Scout

03/06/2024, 2:59 PM
Based on the documentation, when you define a monthly partitions definition using
MonthlyPartitionsDefinition
, it is designed to create partitions that correspond to a time window. For example, if you start with a date like "2022-03-12", it will create partitions such as (2022-04-01-00:00, 2022-05-01-00:00), and so on. When you materialize an asset that uses this partitions definition, the
AssetExecutionContext
provides access to the time window of the current partition through the
partition_time_window
property. This is the default behavior and is intended to provide the bounds of the time window for the current run. However, if you want to access the partition keys directly within a
DBIOManager
, you would typically use the
partition_key
property from the
AssetExecutionContext
. This property gives you the string representation of the partition key for the current run, which is based on the start of the partition's time window. If you are encountering a situation where you are getting a partition range instead of a single partition key, this might be due to the asset being part of a backfill or a similar operation that involves multiple partitions. The documentation mentions that if you want to support running a backfill of several partitions in a single run, you can use the
partition_keys
property to get all of the partitions being materialized by the backfill. In the case of a
DBIOManager
, if it's designed to handle single partitions and you are trying to use it with multiple partitions (such as during a backfill), you might run into issues if the
DBIOManager
is not equipped to handle partition ranges. The documentation does not provide a direct solution for making a monthly partitions definition return partition keys in a
DBIOManager
instead of a partition window. If you are facing a specific issue with the
DBIOManager
and partition keys, it might be a limitation or a use case that is not directly addressed in the documentation. In such cases, it would be advisable to look for any relevant issues or discussions in the Dagster GitHub repository or seek guidance from the Dagster community for potential workarounds or solutions.
3 Views