https://dagster.io/ logo
#ask-community
Title
# ask-community
f

Félix Tremblay

04/21/2023, 1:07 PM
A question about time-based partitions. Is there a way to define custom duration (e.g. 15 minutes), for example, when 1h is too slow.
v

Vinnie

04/21/2023, 1:11 PM
You can use `TimeWindowPartitionDefinition`s with custom cron schedules for arbitrary durations, here’s an example I have for a homegrown
QuarterlyPartitionsDefinition
(which I should add, you can just as easily just create it with the
TimeWindowPartitionDefinition
instead of creating another class, I just did it to keep things a little more consistent)
Copy code
class QuarterlyPartitionsDefinition(TimeWindowPartitionsDefinition):
    def __new__(
        cls,
        start_date: Union[datetime, str],
        cron_schedule: str = "0 0 1 */3 *",
        fmt: Optional[str] = None,
        end_offset: Optional[int] = 0,
    ):
        _fmt = fmt or "%Y-%m-%d"

        return super(QuarterlyPartitionsDefinition, cls).__new__(
            cls,
            start=start_date,
            cron_schedule=cron_schedule,
            fmt=_fmt,
            end_offset=end_offset,
        )
f

Félix Tremblay

04/21/2023, 1:13 PM
Thanks @Vinnie that is exactly what I was looking for!
Thanks @Vinnie that is exactly what I was looking for!
3 Views