Remi Gabillet
03/10/2023, 1:21 PMStaticPartitions
. I'm trying to use define_asset_job
with a schedule specifying the partitions to build but I'm stuck.
If I define the job ignoring partitions as such:
spider_stable_index_job = define_asset_job(
"job",
AssetSelection.keys([...])
)
ScheduleDefinition(job=job, cron_schedule="0 * * * *")
It errors with: Tried to access partition_key_range for a non-partitioned run
If I pass tags to explicitly define the range of partitions:
spider_stable_index_job = define_asset_job(
"job",
AssetSelection.keys([...]),
tags={
'dagster/asset_partition_range_start': '-214',
'dagster/asset_partition_range_end': '214'
}
)
ScheduleDefinition(job=job, cron_schedule="0 * * * *")
I get this error: dagster._check.CheckError: Invariant failed. Description: Attempted to set tag with reserved system prefix: dagster/asset_partition_range_end
If I pass the `PartitionDefinitions`:
spider_stable_index_job = define_asset_job(
"job",
AssetSelection.keys([...]),
partitions_def=static_partitions
)
ScheduleDefinition(job=job, cron_schedule="0 * * * *")
The sensor does not schedule any job.
Is there a way to define an asset job with specific partitions?