https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Justin Beredo

05/11/2022, 2:42 AM
Hi, is there a way to run all static partitions at the same time?
y

yuhan

05/11/2022, 5:10 PM
Hi Justin, a way I can think of is to create a schedule for static partitioned job and launch backfills for all partitions. for example:
Copy code
from dagster import job, op, static_partitioned_config

CONTINENTS = [
    "Africa",
    "Antarctica",
    "Asia",
    "Europe",
    "North America",
    "Oceania",
    "South America",
]


@static_partitioned_config(partition_keys=CONTINENTS)
def continent_config(partition_key: str):
    return {"ops": {"continent_op": {"config": {"continent_name": partition_key}}}}


@op(config_schema={"continent_name": str})
def continent_op(context):
    <http://context.log.info|context.log.info>(context.op_config["continent_name"])


@job(config=continent_config)
def continent_job():
    continent_op()


from dagster import schedule


@schedule(cron_schedule="0 0 * * *", job=continent_job)
def continent_schedule():
    for c in CONTINENTS:
        request = continent_job.run_request_for_partition(partition_key=c, run_key=c)
        yield request


@repository
def my_repo():
    return [continent_schedule, continent_job]
Then, in Dagit UI, you can navigate to the “Partitions” tab and “Launch backfill” for all the partitions.
j

Justin Beredo

05/12/2022, 1:26 AM
Thanks! This seems like the solution. 🙇
2 Views