Hi. We have the following requirement: for our daily Dagster jobs, we need the assets to backfill for X number of days in the past, instead of only materializing the current day. Basically a "rolling" backfill. Our idea was to introduce a new daily job composed of one op that triggers a Dagster backfill of the main assets job (using somehow the internal python modules, not the CLI). But we're wondering if that's even possible in the context of the OSS deployment (Helm chart) to trigger a backfill from a job's pod since the Dagster daemon is not running or accessible there. Any thoughts or recommendations?

Hi Guillaume - one thing to note is that Dagster schedules can return multiple RunRequests from the schedule function, and each of them will be launched on the schedule's cron schedule. So you could potentially implement the backfill that way by returning X RunRequests each tiem? You could also launch a backfill via the Dagster Graphql API (the same way that Dagit does) - but the option above is what i'd try first. The main #dagster-support channel may be a better bet for questions about how to model that exactly

Hi @daniel. Thanks for answering. So am I understanding this correctly: instead of defining a schedule based on a single job, we would define a different schedule based on multiple

RunRequests

? Any chance you could roughly point me to the right API to achieve that?

Sure - here's an example of a schedule that returns multiple RunRequests: https://docs.dagster.io/concepts/partitions-schedules-sensors/schedules#static-partitioned-jobs

I see, so the trick comes down to using "static partitions" which I would generate dynamically based on the date?

The example there was just intended to show multiple run requests - i don't think you would use static partitions for this.

Correct. I want everyday to have the equivalent of a 7 days backfill

ok, I think you would still use the run_request_for_partition function like in that example, but for the time-based partition keys that you want to run

That helps a lot, thanks!