https://dagster.io/ logo
m

Michael Qiu

03/15/2023, 6:58 PM
Hey all, I'm noticing some interesting behavior in the
_get_dbt_op
function here, where the Dagster step that executes the DBT build command succeeds even though the DBT build command may fail (https://github.com/dagster-io/dagster/blob/1ceda3d643e8009ff6a8c16a415375cae804d46[…]/python_modules/libraries/dagster-dbt/dagster_dbt/asset_defs.py). Just a disclaimer that we have our own version of
_get_dbt_op
in our codebase, but it is almost identical to the one here. We ended up having to add an except block in the try/finally + some additional handling here just so we know when to fail the step (https://github.com/dagster-io/dagster/blob/1ceda3d643e8009ff6a8c16a415375cae804d46[…]/python_modules/libraries/dagster-dbt/dagster_dbt/asset_defs.py). I just wanted to verify that this is expected behavior, where the step would succeed despite the DBT build operation failing and throwing an exception. Thanks!
o

owen

03/17/2023, 3:22 PM
hi @Michael Qiu! this is not expected behavior, at least in the default case. there is an "ignore_handled_error" configuration option on the dbt_cli_resource which should toggle this behavior on or off, but it should be set to
False
by default. This is where the return code of the dbt subprocess is handled: https://github.com/dagster-io/dagster/blob/1ceda3d643e8009ff6a8c16a415375cae804d46[…]c/python_modules/libraries/dagster-dbt/dagster_dbt/cli/utils.py What version of dbt are you on? Specifically is it <1.4 or >=1.4?
m

Michael Qiu

03/17/2023, 5:16 PM
We are pre 1.4 right now, but will be upgrading soon. We have a workaround for this for now, but we will test the behavior of this once we upgrade. Thanks for the help!
3 Views