Hi, question on migrating from

load_assets_from_dbt_project

to the new flexible way of building dbt assets from a dbt project using

@dbt_assets

and

DbtCliResource

like in this tutorial. Currently we use the following:

load_assets_from_dbt_project(
    project_dir=DBT_PROJECT_DIR,
    profiles_dir=DBT_PROFILES_DIR,
    node_info_to_group_fn=get_dbt_group_name, -- prefixes the asset group name with "dbt_"
)

I’ve looked at the

1.4.14

docs and it’s not clear to me where we can specify an alternative

profiles.yml

path in the new way:

@dbt_assets(manifest=dbt_manifest_path)
def jaffle_shop_dbt_assets(context: OpExecutionContext, dbt: DbtCliResource):
    yield from dbt.cli(["run"], context=context).stream()
    yield from dbt.cli(["test"], context=context).stream()

With regards to the

node_info_to_group_fn

, is that now replaced with the kwarg of

dagster_dbt_translator

in

@dbt_assets(….)

? Where the value is a DagsterDbtTranslator object specifically like this example?

Here is an example for group_fn https://github.com/dagster-io/hooli-data-eng-pipelines/blob/master/hooli_data_eng/assets/dbt_assets.py#L42 For profiles-dir, I believe the default is to look within the main dbt project dir (same as the dbt cli default), but if you want to pass in that arg to every invocation, here is an example of how you could write a lightweight extension of the dbtCliResource to do so: https://github.com/dagster-io/hooli-data-eng-pipelines/blob/master/hooli_data_eng/resources/dbt.py The project shows how to reference that resource and configure it as well

Thanks for the links Sean. Looking at the 2nd link, it seems like I could also achieve this with the following…?

@dbt_assets(manifest=dbt_manifest_path)
def jaffle_shop_dbt_assets(context: OpExecutionContext, dbt: DbtCliResource):
    yield from dbt.cli(["run" ,"--profiles-dir /my/custom/dir"], context=context).stream()
    yield from dbt.cli(["test", "--profiles-dir /my/custom/dir"], context=context).stream()

I would expect that to output a valid CLI command that can be run. I would like to avoid extending the main dagster cli class and try to keep things as “out the box” and vanilla as possible

Believe so yes, you’ll just need to remember to add that to every one

You can follow this: https://dagster.slack.com/archives/C04CW71AGBW/p1690578876840239?thread_ts=1690384210.355069&cid=C04CW71AGBW if you put your profiles.yml in the root of your dbt project dir, then you don’t need to configure it as a CLI flag for every command.

Thanks! I’ve managed to do both blob highfive. The profiles dir i’m currently passing as an arg, but will move away from that and set the env variable. With regards to the

CustomDagsterDbtTranslator

example, that helped a lot. I should add that the example here would ideally need the

@classmethod

decorator so it matches the exact same function body as

DagsterDbtTranslator

otherwise mypy will complain when doing static type checking.

This will be first class in this week’s release once https://github.com/dagster-io/dagster/pull/15722 is landed. Thanks for the feedback.