https://dagster.io/ logo
#ask-ai
Title
# ask-ai
w

Wonjae Lee

02/15/2024, 5:14 PM
is it possible to update partitions before the run through dynamic partition but not based on sensor?
s

Scout

02/15/2024, 5:14 PM
Based on the documentation, dynamic partitioning in Dagster is typically managed through sensors or schedules. Sensors can be used to populate dynamic partitions, and they can be triggered by various events, such as changes in upstream assets. The documentation also mentions the use of
AddDynamicPartitionsRequest
and
DeleteDynamicPartitionsRequest
which can be evaluated by a sensor or schedule to manage dynamic partitions. However, there is a discussion in the community about the possibility of utilizing dynamic partitioning without relying on sensors or schedules, specifically through the use of
execute_in_process
. The documentation does not explicitly state a method for updating dynamic partitions before a run without using sensors or schedules, and the conversation suggests that this is an area of ongoing development and exploration within the Dagster community. Therefore, while the typical approach involves sensors or schedules, if you are looking for alternative methods to manage dynamic partitions, it may be worth exploring community discussions or reaching out to the Dagster team for more current or upcoming features that might address this use case.
5 Views