https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jakub Zgrzebnicki

12/01/2022, 7:11 AM
Hi all, I need to make 15min interval partitioned job that reach around 1 year back. That is around 35040 partitions which is problematic to handle by dagster dagit in partition tab that hangs or loads quite long. I thought by dynamicaly generating jobs by quarters to handle only 3 months at a time, but from what I know there will be problem on the ends of the quarters, as jobs are only generated at repo reload and it may happend that new job and schedule won't be generated. Is there a way to handle such amounts of partitions?