paul.q
08/10/2023, 4:39 AMclaire
08/10/2023, 10:27 PM2023-01-01
). This means that it's not currently possible to identify whether a materialization with a 2023-01-01
key materialized for the whole month or just one day.
If you primarily want to be able to have large partitions for more efficient backfills, I recommend you instead use single run backfills: https://docs.dagster.io/concepts/partitions-schedules-sensors/backfills#single-run-backfills
Using single run backfills, a single run will be kicked off for each partition range, and assuming your IO manager can operate on partition ranges, a single output will be persisted.