https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jason Myers

08/09/2023, 8:05 PM
I'm running into an issue with Dagster + DBT where I can run something in the container directly but through dagster it fails and I can't get good logs for it
image.png
image.png
image.png
I manually ran the command in that same container the job ran in
Copy code
@dbt_assets(
    manifest=DBT_MANIFEST,
    select="staged__nc_op_asc_patients",
    dagster_dbt_translator=CustomDagsterDbtTranslator(),
)
def hca_nc_op_asc_dbt(
    context: OpExecutionContext,
    config: StateDataProcessingConfig,
    dbt_cli: DbtCliResource,
    upstream_process,
    overlapping_periods,
):
    all_periods = set(overlapping_periods["append"]).union(set(overlapping_periods["replace"]))
    dbt_vars = {
        "replace_reporting_periods": str(overlapping_periods["replace"]),
        "all_reporting_periods": str(list(all_periods)),
    }
    dbt_args = ["run", "--vars", f"'{json.dumps(dbt_vars)}'", "-s", "staged__nc_op_asc_patients"]

    dbt_cli_task = dbt_cli.cli(dbt_args, context=context)

    yield from dbt_cli_task.stream()
r

rex

08/09/2023, 8:22 PM
j

Jason Myers

08/09/2023, 8:51 PM
they are empty, even while the container is alive
o

owen

08/09/2023, 10:12 PM
hi @Jason Myers -- this is definitely odd behavior, but one more attempt to find some logs might be to look in
/opt/.../dbt_project/logs/dbt.log
, which is the default path that dbt writes its own logs to.
j

Jason Myers

08/10/2023, 12:37 AM
thank you will check in the AM
nothing that was helpful there, I believe it's caused by a mix of v1 and v2 dbt clis in the same project, changing it up
o

owen

08/10/2023, 8:53 PM
gotcha -- that sounds like a reasonable diagnosis
r

rex

08/10/2023, 8:55 PM
How did you deploy your project? Similar to this other user, you may not be seeing your stdout logs in the compute logs if you did not set up a compute log manager: https://dagster.slack.com/archives/C04CW71AGBW/p1691682731139089?thread_ts=1691679416.046079&cid=C04CW71AGBW
j

Jason Myers

08/13/2023, 7:15 PM
I'm seeing this happen locally I think it might be when the code location and dagster/dagit are on different versions