Hi I'm getting an error when running dbt via dagst...
# integration-dbt
Hi I'm getting an error when running dbt via dagster:
Copy code
dagster_dbt.errors.DagsterDbtCliOutputsNotFoundError: Expected to find file at path /opt/dagster/app/mydbt/assets/../../mydbt/target/run_results.json
 File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_plan.py", line 265, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 382, in core_dagster_event_sequence_for_step
    for user_event in check.generator(
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 94, in _step_output_error_checked_user_event_sequence
    for user_event in user_event_sequence:
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/compute.py", line 177, in execute_core_compute
    for step_output in _yield_compute_results(step_context, inputs, compute_fn):
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/compute.py", line 145, in _yield_compute_results
    for event in iterate_with_context(
  File "/usr/local/lib/python3.8/site-packages/dagster/_utils/__init__.py", line 457, in iterate_with_context
    next_output = next(iterator)
  File "/usr/local/lib/python3.8/site-packages/dagster_dbt/asset_defs.py", line 425, in _dbt_op
    dbt_output = DbtOutput(result=dbt_resource.get_run_results_json())
  File "/usr/local/lib/python3.8/site-packages/dagster_dbt/cli/resources.py", line 321, in get_run_results_json
    return parse_run_results(project_dir, target_path)
  File "/usr/local/lib/python3.8/site-packages/dagster_dbt/cli/utils.py", line 171, in parse_run_results
    raise DagsterDbtCliOutputsNotFoundError(path=run_results_path)

The above exception occurred during handling of the following exception:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/dagster/app/mydbt/assets/../../mydbt/target/run_results.json'
File "/usr/local/lib/python3.8/site-packages/dagster_dbt/cli/utils.py", line 168, in parse_run_results
    with open(run_results_path, encoding="utf8") as file:

The above exception occurred during handling of the following exception:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 117: invalid continuation byte

File "/usr/local/lib/python3.8/site-packages/dagster_dbt/asset_defs.py", line 420, in _dbt_op
    dbt_output = dbt_resource.run(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/dagster_dbt/cli/resources.py", line 145, in run
    return self.cli("run", models=models, exclude=exclude, select=select, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/dagster_dbt/cli/resources.py", line 90, in cli
    return execute_cli(
  File "/usr/local/lib/python3.8/site-packages/dagster_dbt/cli/utils.py", line 107, in execute_cli
    line = raw_line.decode("utf-8").rstrip()
I'm not sure were to search in my dbt project were it is creating 0xf0
daggy love 1
How do I ensure the encoding is correct.
hi @Benedikt Buchert! this is a compatibility issue between
dbt  1.4.x
this will be fixed in this week's release, but until then if you downgrade dbt to < 1.4, this error should be resolved
👍 1
Thank you I will downgrade dbt and test again.
Worked like a charm. Thank you!
@owen I’m still receiving this error. Using dbt-bigquery==1.0.0
Any idea how I could resolve it?
hi! sorry about this -- the compatibility fix slipped to this week's release. the same fix applies here (just pinning dbt to < 1.4) but we have a fix tested and ready to merge for this Thursday
@owen This still doesn’t seem to be working for me, the error persists
Copy code
import glob
from setuptools import find_packages, setup

    # package data paths are relative to the package key
        "assets_dbt_python": ["../" + path for path in glob.glob("dbt_project/**", recursive=True)]
        # packaging v22 has build compatibility issues with dbt as of 2022-12-07
        # fixing dbt versions due to dagster conflict
    extras_require={"dev": ["dagit", "pytest"]},
ah that looks like a different error -- are there any logs from the dbt execution that indicate what the root cause might be?
Looks like it’s a dagster_dbt specific error
just wraps the underlying dbt execution error ("`could not find command, ensure it is in the user's PATH: "gcloud"` ) -- looks like
is not available in your environment (might need to add that to your setup.py)
Okay, thanks for that pointer. I’m using environment variables for the profiles.yml parameters. I have a suspicion that might be the issue
Is there any recommendation on how to correctly set-up the profile parameters? https://docs.dagster.io/integrations/dbt/using-dbt-with-dagster/part-one Doesn’t seem to recommend a specific approach
as in, you're using
{{ env(...) }}
inside your profiles.yml? In general, I wouldn't expect that to cause any issues as long as those env variables are defined in the environment you're running in (I've used that setup before with no issues).