Hello noob question what is the recommended way to associate dagster #ask-community

Hello - noob question: what is the recommended way...

Miguel Rodriguez

06/28/2023, 12:03 AM

Hello - noob question: what is the recommended way to associate monthly partitioned assets to run on a weekly or daily schedule. In other words: I would like to recreate the assets daily but the assets themselves correspond to a month worth of data? This schedule works great but will only run once a month:

Copy code

monthly_assets_schedule = build_schedule_from_partitioned_job(
    define_asset_job(
        "monthly_assets_job", selection=AssetSelection.groups("core_monthly")
    )
)

dagster bot surface to discussion 1

Miguel Rodriguez

06/28/2023, 3:31 AM

I think this worked! But, I am still wondering if this is the recommended approach.

Copy code

monthly_partitions_def = MonthlyPartitionsDefinition(start_date="2023-01-01")

monthly_assets_job = define_asset_job(
    "monthly_assets_job", selection=AssetSelection.groups("core_monthly")
)


@schedule(cron_schedule="@daily", job=monthly_assets_job)
def monthly_assets_schedule():
    return RunRequest(partition_key=monthly_partitions_def.get_last_partition_key())

sean

06/28/2023, 12:30 PM

Hi Miguel, Yes you’ve hit on the right approach.

build_schedule_from_partitioned_job

is just sugar to create a schedule with the same cadence as a partitions def you’ve already defined, but if you need a different cadence you should just fall back to

@schedule

Miguel Rodriguez

06/28/2023, 3:41 PM

Awesome. Thanks @sean

Open in Slack

Previous Next