How can i

group

dynamic partitions which are adjacent event timestamps into a downstream model with a time-based partition such as @daily or @weekly ? I have a source (S1) that is being refreshed as-needed during the day (dynamic partitions as timestamps) and I want the downstream dependency to run a daily report (R1) using a time-based partition @daily. So for a given day, the R1 partition depends on all the S1 partitions in that day range.

hi @Johno Scott! this is an interesting use case, but this sort of dynamic partition mapping is not currently supported. would you mind filing an issue for this? cc @claire who I believe was thinking about something similar

Hi Johno! Here's a link to an issue that more generally explores custom mappings for dynamic partitions: https://github.com/dagster-io/dagster/issues/13139 Some thoughts we've played around with: • having some sort of regex function to map partitions to each other • having some sort of yaml file that defines the partitions that are mapped to one another Though for your use case it might make sense to have a function defined in dagster that basically converts a timestamp into a time partition window

@owen I will file an issue.