I have an `asset job and schedule` defined almost exactly ac dagster #ask-community

I have an `asset, job, and schedule` defined almos...

Drew You

07/04/2023, 5:52 PM

I have an

asset, job, and schedule

defined almost exactly according to the partitioned asset tutorials and I'm getting the following error:

dagster._core.errors.DagsterInvariantViolationError: Cannot access partition_key for a non-partitioned run

. Code below.

🤖 1

dagster bot responded by community 1

Drew You

07/04/2023, 5:52 PM

asset:

Copy code

@asset(
    required_resource_keys={"duckdb"},
    partitions_def=fund_structure_partition,
)
def my_asset(context):
  context.partition_key
  ...

Jacob Marcil

07/04/2023, 5:54 PM

Can you copy the configuration of the job and schedule too please?

Jacob Marcil

07/04/2023, 5:56 PM

Try

context.asset_partition_key_for_output()

Drew You

07/04/2023, 5:56 PM

job:

Copy code

asset_job = define_asset_job(
    "asset",
    selection=["group/my_asset"],
    partitions_def=fund_structure_partition,
)

Drew You

07/04/2023, 5:56 PM

yeah, I tried that first. had the same error

Jacob Marcil

07/04/2023, 5:57 PM

selection=["group/asset"],

should point to the asset name. if that case

group/my_asset

I think.

Drew You

07/04/2023, 5:59 PM

oh, right it does. just a typo on my part

Drew You

07/04/2023, 6:00 PM

Copy code

my_asset_schedule = ScheduleDefinition(
    job=asset_job,
    cron_schedule="*/5 * * * *",
    default_status=DefaultScheduleStatus.RUNNING,
)

Jacob Marcil

07/04/2023, 6:02 PM

Ohh so probably the schedule. Try this.

Copy code

my_asset_schedule = build_schedule_from_partitioned_job(
    asset_job,
    default_status=DefaultScheduleStatus.RUNNING,
)

I think you need a partitionned schedule in order to run a partition job. I can be wrong, be using

build_schedule_from_partitioned_job()

with all my assets works fine.

Drew You

07/04/2023, 6:04 PM

oh, hmm

Drew You

07/04/2023, 6:05 PM

fund_structure_partition = StaticPartitionsDefinition(list(FUND_STRUCTURE.keys()))

Drew You

07/04/2023, 6:05 PM

so, I think

build_schedule_from_partitioned_job

only works for `TimeWindowPartition`s

Jacob Marcil

07/04/2023, 6:10 PM

When I work with Static Partitions I usually issue run request directly instead of using

ScheduleDefinition

. Looking at the documentation event with static partitions they configure it by issuing

RunRequest

so I think you should try this. https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions#creating-schedules-from-partitioned-jobs https://docs.dagster.io/concepts/partitions-schedules-sensors/schedules#schedules-that-provide-custom-run-config-and-tags

Jacob Marcil

07/04/2023, 6:12 PM

This is the end of my knowledge in static partition jobs. Hope this worked. If not someone from Dagster will surely answer this thread soon. 🙂

4 Views

Open in Slack

Previous Next