Hi! We retrieve data daily via an API that we transform before writing it to a DWH. It happens sometimes that a change has been made on the API which implies that we need a complete history (over a time range of 10 years for example). Some APIs are limited and we cannot run the 10 * 365 executions in parallel, the rebuilding time is much too long using daily partitions. That's why we define a second job that uses monthly partitions to rebuild the history (i.e. 10*12 executions) and it works quite well. The constraint of this usage is that we have to double all our assets. So we have a lot of assets in the same code location and I don't find it ideal to have several assets that materialize the same data but at different levels of granularity. Is there any other alternative to rebuild a history that would not require duplicate assets?

Hi Jordan - Is this an issue with the time that its taking to backfill?