https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
a

Alex Orlovskyi

08/20/2023, 1:56 PM
Hi Dagster team and folks! Trying to update: • dagster: v1.3.13 -> 1.4.7 • dbt: v1.4.5 -> 1.6.0 Trying to use new dagster-dbt API. Consider following snippet:
Copy code
@dbt_assets(
    manifest=Path(settings.SOME_DBT_PROJECT_PATH, "target", "manifest.json"),
    partitions_def=some_partitions_def,
    select="some.model",
    dagster_dbt_translator=SomeDagsterDbtTranslator(),
)
def some_dbt_assets(
    context: OpExecutionContext,
    dbt: DbtCliResource,
    config: SomeDBTAssetConfig,
):
    dbt_invocation = dbt.cli(["run", "--vars", json.dumps(config.dict())], context=context)

    yield from dbt_invocation.stream()

    run_results_json = dbt_invocation.get_artifact("run_results.json")
    <http://context.log.info|context.log.info>(f"dbt run results: {run_results_json}")
With this code when I have some problems with dbt, like SQL error I see following state at the Dagster UI (attached screenshot). Question: will dbt errors (like sql error in model) be parsed and displayed at error context at Dagster UI as it was previously (in my case - when dagster was v1.3.13, dbt 1.4.5)? If no, then -> how can i retrieve error details without need to access file
run_results.json
manually? (Tried to do so programmatically (last 2 rows in my code snippet), but Dagster is calling
_raise_on_error
inside
.get_artifact
so execution don't even come to that point).
dagster bot responded by community 1
🤖 1
q

Qwame

08/20/2023, 4:40 PM
You'd have to access the raw computer logs. Those errors do not show in the event logs anymore.
The button on the right in the pink box will give you the compute logs
🎉 1
ty spinny 1
a

Alex Orlovskyi

08/20/2023, 5:11 PM
@Qwame thx for help)
r

rex

08/21/2023, 3:12 PM
Thanks Qwame for helping out Alex. Looks like being unable to see the debug logs when needed, or not knowing how to access the Dagster compute logs has been a recurring issue. In order to rectify this, I’ve now updated the error message here: https://github.com/dagster-io/dagster/pull/15958. We’ll now provide a link the
dbt.log
file produced by your computation, so that you can inspect that quickly when doing local development. In production though, we’ll still recommend that you view your error logs in our Dagster compute logs, as Qwame mentioned.
ty spinny 1
a

Ankit Singhal

09/21/2023, 4:11 PM
@Gaurav Lathwal
2 Views