Hi! After reviewing the documentation on partition...
# ask-community
e
Hi! After reviewing the documentation on partitioning assets I'm not seeing a built in way to partition assets by year (only hourly-monthly). What's the best way to accomplish this? Thanks!
t
Haven't tried this myself, but you'll likely have to write your own Time Window Partition. You might able to build off and use the classes provided in the out-of-the-box partitions for reference though. ex. here, https://github.com/dagster-io/dagster/blob/62118c151593fdf927c29d6ac18d0bc62c53545[…]les/dagster/dagster/_core/definitions/time_window_partitions.py
D 1
If you do this though, I'd love and wholeheartedly support if you rose a PR back to the Dagster project for this!
D 1
c
Open issue here for this: https://github.com/dagster-io/dagster/issues/12415 In the meantime, you could use
TimeWindowPartitionsDefinition(cron_schedule="0 0 1 1 *")
daggy 3d 1
💯 1
d
@claire I was wondering if we could create a
CronPartitionsDefinition
. This would allow users to have the flexibility to choose any partition time partitions they need. Also, a
cron_partitioned_config
decorator can be created to ensure consistency with other partitioned configs. Here is the code snippet that outlines the features:
c
Hi @Derek Chang. To my understanding,
CronPartitionsDefinition
and
build_schedule_from_cron_partitioned_job
are just wrappers around the existing
TimeWindowPartitionsDefinition
and
build_schedule_from_partitioned_job
functions, right? I think it makes sense to introduce
CronPartitionsDefinition
and
cron_partitioned_config
for symmetry with
HourlyPartitionsDefinition
,
hourly_partitioned_config
etc. Happy to review PRs if you have the bandwidth to contribute. I think we don't need to support
build_schedule_from_cron_partitioned_job
since we don't have similar functions for hourly partitioned jobs, daily partitioned jobs, etc.