:wave: Hello! I’m completely new to dagster and ha...
# ask-community
👋 Hello! I’m completely new to dagster and have mostly only read documentation. So I’m sorry if this is mentioned anywhere that I missed. I have a question regarding backfilling. Say I have a daily partitioned asset defined that requests the data using a http request and produces a file to disk, say using parquet. I’d like to backfill this for a whole year. From my understanding, that would make one HTTP request per partition. But the API I’m using supports larger batches, say per month. How would I go about backfilling a daily partitioned asset, for a full year, using monthly requests (12 HTTP calls for 365 files)?
Hey @Philippe Laflamme- we don't currently have the ability to break up a backfill into multiple runs that each operate on a range, but you can launch a single run that operates on a range. I just wrote up a short guide on how to do this: https://github.com/dagster-io/dagster/discussions/11653. Here's an issue where we're tracking more general requests on this topic: https://github.com/dagster-io/dagster/issues/8706
Awesome, thanks!