Hi all I encounter a problem with define asset job and confi dagster #ask-community

Hi all, I encounter a problem with define_asset_j...

Xavier BALESI

07/26/2022, 11:23 AM

Hi all, I encounter a problem with define_asset_job and config a little bit similar to https://dagster.slack.com/archives/C01U954MEER/p1658751578283169 but not exactly the same. When I define asset job in the code like that :

Copy code

asset_down_job = define_asset_job(
    name="asset_downa_job",
    selection="down_asset",
    description="down asset job",
)

and launch with dagit with the appropriated config :

Copy code

resources:
  s3_io_manager:
    config:
      s3_bucket: mybucket
      s3_prefix: myprefix
  s3:
    config:
      profile_name: myprofile

All is OK but when I put exactly the same config in a file and set it in the

define_asset_job

function :

Copy code

asset_down_job = define_asset_job(
    name="asset_downa_job",
    selection="down_asset",
    description="down asset job",
    config=config_from_files(["asset_downa_job.yaml"]),
)

dagster raises an error :

Copy code

dagster.core.errors.DagsterInvalidConfigError: Error in config when building job 'asset_downa_job' from graph 'asset_downa_job' 
    Error 1: Missing required config entry "ops" at the root. Sample config for missing entry: {'ops': {'down_asset': {'inputs': {'up_asset': '<selector>'}}}}

Do I a mistake or is it a bug ? Thank you for your help

Xavier BALESI

07/26/2022, 11:58 AM

I tried to fill the required config with :

Copy code

ops:
  down_asset:
    inputs:
      up_asset:
        pickle:
          path: up_asset

The result is an error in config job that notify "inputs" is unexpected :

Copy code

dagster.core.errors.DagsterInvalidConfigError: Error in config for job
    Error 1: Received unexpected config entry "inputs" at path root:ops:down_asset. Expected: "{ config?: { } outputs?: [{ result?: { json: { path: String } pickle: { path: String } } }] }".

owen

07/28/2022, 10:39 PM

hi @Xavier BALESI! This was a bug that was fixed in today's release, so if you upgrade the newest dagster version, this should be fixed 🙂

Xavier BALESI

07/29/2022, 7:12 AM

Wonderful ! thank you @owen. I will test today 👍

66 Views

Open in Slack

Previous Next