https://dagster.io/ logo
#ask-ai
Title
# ask-ai
d

Deividas Jodogalvis

02/06/2024, 2:07 PM
How to execute backfill with python apis?
s

Scout

02/06/2024, 2:07 PM
To execute a backfill for a job using Python APIs, you can use the
dagster job backfill
command provided by Dagster's command-line interface (CLI). Here's how you can do it: 1. Backfilling all partitions in a job: If you have a date-partitioned job named
trips_update_job
, you can execute the backfill for this job by running the following command in your terminal:
Copy code
shell    $ dagster job backfill -p trips_update_job
This command will display a list of all the partitions in the job, ask you if you want to proceed, and then launch a run for each partition. 2. Backfilling a subset of partitions: To execute a subset of a partition set, you can use the
--partitions
argument and provide a comma-separated list of partition names you want to backfill:
Copy code
shell    $ dagster job backfill -p do_stuff_partitioned --partitions 2021-04-01,2021-04-02
Alternatively, you can also specify ranges of partitions using the
--from
and
--to
arguments. Please note that the above examples are for using the CLI to launch backfills. Currently, there is an open feature request on Dagster's GitHub repository for creating a Python API for launching backfills for a job, which suggests that this functionality may not be available directly through Python APIs yet. You can track the progress or contribute to the discussion on the feature request issue on GitHub.
2 Views