Hey, I'm seeing this when clicking materialise on a specific asset in a pipeline. If I go to the launchpad for the pipeline it tells me all config is present and correct. If I run the entire pipeline it also runs successfully. Google is not being kind on this one

I'm seeing the same behavior and am similarly a bit confused... are you grouping these assets using

load_assets_from_modules

?

Phew, not only me! 😅 I have workspace.yml like this

load_from:
  - python_package: tmp

and

tmp/__init__.py

has an

@repository

definition. some of my assets are defined as

@asset

some from

AssetsDefinition.from_graph

..now that you mention it, I think it's the

AssetsDefinition.from_graph

that are having this issue

hey Oliver - are you supplying a config value for

define_asset_job

? or manually specifying config in the launchpad at the time you launch the job?

I am using

build_asset_selection_job

dev_asset_job = build_asset_selection_job("dev", assets=resourced_assets, source_assets=[],        config=yaml.safe_load(Path("tmp/dev.yaml").read_text()),
        resource_defs=dev_resource_defs,
)

Ah, I see what's going on. This is because you've supplied the config for the job, but not for the assets themselves - that means it will apply when you run the job, but not when you materialize the assets independently from the job. You could do something like:

@repository
def dev_repo():
    resource_config = yaml.safe_load(Path("tmp/dev.yaml").read_text())
    return [with_resources(assets, dev_resource_defs, resource_config), define_asset_job("dev", selection=AssetSelection.assets(assets))]

By the way, any particular reason to use

build_asset_selection_job

instead of

define_asset_job

? The former is an internal API that could change in a future release.

Ah ok thanks.

build_asset_selection_job

allows you to specify the executor while

define_asset_job

does not which is why I'm using it. Could switch that to be defined on the repo but it's a little awkward for the workflow im developing. I have one job that uses local multiprocessing executor for quick dev and testing. Then I have another job that uses a custom executor to run pipelines on infra in a cloud environment to test on more large scale datasets

I have one job that uses local multiprocessing executor for quick dev and testing. Then I have another job that uses a custom executor to run pipelines on infra in a cloud environment to test on more large scale datasets

Got it - do you want these jobs to show up in Dagit at the same time? Our typical recommendation is to put these jobs in different dagster repos, so that you can target one with your production setup and another with your local dev setup.

oh I see, yup.. nice thanks 🙂