https://dagster.io/ logo
#ask-community
Title
# ask-community
h

Harpal

07/18/2022, 3:08 PM
Has anyone used the
load_assets_from_dbt_manifest
method to load dbt assets before? Upon switching from
load_assets_from_dbt_project
one of my jobs has broken 😞 One of them works without fail (utilises the
load_assets_from_dbt_manifest
method) but the second one (which uses the same method on a different dbt DAG) fails every time. Here is the implementation:
Copy code
manifest_file = f"{DBT_PROJECT_DIR}/target/manifest.json"
with open(manifest_file, "r", encoding="utf-8") as f:
    manifest_json = json.load(f)

dbt_assets = load_assets_from_dbt_manifest(
    manifest_json=manifest_json,
    select="tag:not_rq",
)
The job is simply unable to identify any tables in my database (every single SQL file is unable to SELECT anything
FROM
a_table) See the full error message in the comments.
Error message fro the job that fails. See .sql failures.
j

jamie

07/18/2022, 4:05 PM
cc @owen
h

Harpal

07/18/2022, 4:26 PM
To clarify. When I run this locally (from my laptop but with DBT pointing to my production DB) it works fine. Nothing fails. Here is that dbt command that works on my laptop:
Copy code
dbt run --project-dir ./dbt --profiles-dir ./dbt/config -s tag:not_rq
The equivalent fails on dagster. So I don’t think this is a DBT problem. It may be more of a dagster trying to load DBT elements problem.
j

jamie

07/18/2022, 4:34 PM
thanks for all this info! I'm not very well versed in dbt so i've added owen to this thread and he should be able to take a look
h

Harpal

07/18/2022, 4:48 PM
No problemo folks. Just trying to provide as much context as possible to help get to the bottom of it. Look forward to hearing from yall soon! dagster spin
🎉 1
Oooh I think i’ve solved it! It may have been an issue of not being connected to the right database. I’m testing it from my prod K8s cluster now and will report back with any updates ASAP.
o

owen

07/18/2022, 4:52 PM
haha I was just about to type out a few questions but seems like that's not necessary anymore
h

Harpal

07/18/2022, 4:52 PM
dabparrot
o

owen

07/18/2022, 4:53 PM
glad it seems like you've got it working but definitely check back in if not
now to go take a look at the other thread 😛
🙌 2
3 Views