The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

I'm running into an issue with Dagster + DBT where I can run something in the container directly but through dagster it fails and I can't get good logs for it

image.png

I manually ran the command in that same container the job ran in

```@dbt_assets(
    manifest=DBT_MANIFEST,
    select="staged__nc_op_asc_patients",
    dagster_dbt_translator=CustomDagsterDbtTranslator(),
)
def hca_nc_op_asc_dbt(
    context: OpExecutionContext,
    config: StateDataProcessingConfig,
    dbt_cli: DbtCliResource,
    upstream_process,
    overlapping_periods,
):
    all_periods = set(overlapping_periods["append"]).union(set(overlapping_periods["replace"]))
    dbt_vars = {
        "replace_reporting_periods": str(overlapping_periods["replace"]),
        "all_reporting_periods": str(list(all_periods)),
    }
    dbt_args = ["run", "--vars", f"'{json.dumps(dbt_vars)}'", "-s", "staged__nc_op_asc_patients"]

    dbt_cli_task = dbt_cli.cli(dbt_args, context=context)

    yield from dbt_cli_task.stream()```


the dbt_cli is basically the same thing as <https://github.com/dagster-io/hooli-data-eng-pipelines/blob/master/hooli_data_eng/resources/dbt.py>

You can see the full logs if you check the compute logs: <https://dagster.slack.com/archives/C04CW71AGBW/p1688743197057419?thread_ts=1688742260.619199&amp;cid=C04CW71AGBW>

they are empty, even while the container is alive

hi <@U05KCM9KDPH> -- this is definitely odd behavior, but one more attempt to find some logs might be to look in `/opt/.../dbt_project/logs/dbt.log`, which is the default path that dbt writes its own logs to.

nothing that was helpful there, I believe it's caused by a mix of v1 and v2 dbt clis in the same project, changing it up

gotcha -- that sounds like a reasonable diagnosis

How did you deploy your project? Similar to this other user, you may not be seeing your stdout logs in the compute logs if you did not set up a compute log manager: <https://dagster.slack.com/archives/C04CW71AGBW/p1691682731139089?thread_ts=1691679416.046079&amp;cid=C04CW71AGBW>

I'm seeing this happen locally I think it might be when the code location and dagster/dagit are on different versions