Can somebody point me to the point in the code where

dagster-dbt

matches sources to assets, please? I'm having issues with

source

references in my

dbt

models. I merely want them for lineage tracking. A model (or a model with a

ref

to a model that uses the

source

upstream) uses

{{ source("example_db", "example_table") }}

, which exists in

sources.yml

. No

meta.dagster.asset_key

field. Assume the profile is called

christian

. Executing a job sometimes yields

Failed to start

due to

KeyError: 'source_christian_example_db_example_table'
  File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/impl.py", line 137, in core_execute_run
    yield from execute_run_iterator(
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/api.py", line 863, in __iter__
    yield from self.execution_context_manager.prepare_context()
  File "/usr/local/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 491, in generate_setup_events
    obj = next(self.generator)
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/context_creation_job.py", line 291, in orchestration_context_event_generator
    context_creation_data = create_context_creation_data(
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/context_creation_job.py", line 116, in create_context_creation_data
    resource_keys_to_init=get_required_resource_keys_to_init(execution_plan, job_def),
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/resources_init.py", line 372, in get_required_resource_keys_to_init
    get_required_resource_keys_for_step(job_def, step, execution_plan)
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/resources_init.py", line 405, in get_required_resource_keys_for_step
    input_def = node_def.input_def_named(step_input.name)
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/definitions/node_definition.py", line 143, in input_def_named
    return self._input_dict[name]

It's also flaky - sometimes it seems to work, sometimes it doesn't. It also doesn't affect all jobs, which makes no sense, since I code-gen my

sources.yml

, i.e. if a table exists, it's in there. The format of the missing asset key is

source_${PROFILE}_${SOURCE_DB}_${SOURCE_TABLE}

In my case, the source asset does exists, albeit simply called

example_db / example_table

. The lineage in the UI works, too. Specifying

- name: example_table
    meta:
      dagster:
        asset_key:
        - source_christian_example_db_example_table

Seems to do the trick, but I guess I'll find out on Tuesday after it had a chance to run. More importantly, I don't understand why (and why it sometimes works and sometimes doesn't). So any pointers would be appreciated. fwiw, this is running on K8s + redis. I can't reproduce this locally, Running 1.4.11 + 0.20.11.

Hi @Christian Hollinger - have you seen this? https://docs.dagster.io/integrations/dbt/reference#customizing-asset-keys

Yes, but even when using that to explicitly rename assets, I'm still being hit with a "KeyError", despite the asset existing with exactly that name now.

Mind posting the error you’re hitting, as a new top-level thread in this channel? I’m out for the next couple weeks, and that will allow someone else to get notified when the workweek starts