https://dagster.io/ logo
#ask-ai
Title
# ask-ai
a

Alejandro A

02/26/2024, 8:35 AM
@Scout I have a few assets, that are connected between then and partitioned by a custome time partition. I want them to run into a fix schedule and that the partition gets materialized, how can achieve that?
s

Scout

02/26/2024, 8:35 AM
To achieve a fixed schedule for materializing partitioned assets, you can use Dagster's scheduling system to define a schedule that targets the specific partitions of your assets. Here's a step-by-step guide based on the documentation: 1. Define Your Assets with Partitions: First, ensure that your assets are defined with a custom time partition. You can use the
partitions_def
argument in the
@asset
decorator to specify your custom partitioning scheme. For example, if you have a custom monthly partition, you might define your asset like this:
Copy code
python
from dagster import asset, MonthlyPartitionsDefinition

@asset(partitions_def=MonthlyPartitionsDefinition(start_date="2023-03-01"))
def my_asset():
    # Your asset code here
    ...
2. Create a Partitioned Asset Job: Next, define a job that materializes your partitioned assets. You can use the
define_asset_job
function to create a job that targets the assets you want to materialize.
Copy code
python
from dagster import define_asset_job, AssetSelection

partitioned_asset_job = define_asset_job(
    name="my_partitioned_asset_job",
    selection=AssetSelection.assets(my_asset),  # Replace with your actual assets
    # Specify the same partitions_def as your assets if needed
)
3. Define a Schedule: Now, define a schedule that will trigger the job at fixed intervals. You can use the
build_schedule_from_partitioned_job
function to create a schedule from a partitioned job. This function will automatically create a schedule that materializes the latest partition for each scheduled execution.
Copy code
python
from dagster import build_schedule_from_partitioned_job

asset_partitioned_schedule = build_schedule_from_partitioned_job(
    partitioned_asset_job,
    # You can specify additional configuration here if needed
)
4. Configure the Schedule's Timing: Set the
cron_schedule
parameter to define the fixed schedule timing. For example, if you want to run the job every day at midnight, you would set
cron_schedule="0 0 * * *"
. 5. Deploy and Run: After defining your assets, job, and schedule, deploy your Dagster repository. The schedule will automatically trigger runs according to the cron schedule you've set, materializing the appropriate partitions for each asset. Remember that each schedule tick of a partitioned job fills in the latest partition in the partition set that exists as of the tick time. If you need to adjust which partition is considered the latest, you can use the
end_offset
parameter in your partition definition to shift the partitions forward or backward relative to the schedule tick. Please note that the code examples provided are based on the documentation and should be adapted to fit your specific use case and partitioning scheme. If you have any furth
er questions or need assistance with debugging, feel free to ask!