https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
b

Benjamin Faught

06/12/2023, 1:59 PM
Hi! I've been working through a POC of Dagster for my team so apologies if this is clear to everyone else... I'm wondering if there is a way to set up the dbt resource and reference it when setting up the assets? For example, is there a way to set up the dbt resource in the resources folder and then reference it here vs specifying it here as well?
q

Qwame

06/12/2023, 2:32 PM
What version of dagster are you using? I am sure you do not need the
with_resources
in the current API. You just pass the resource to
Definitions
and you should be able to reference it from any other asset
b

Benjamin Faught

06/12/2023, 4:49 PM
Ahh I'm on 1.3.7. Perhaps that's the issue then? If I set a dbt resource in a resource file with other ones like snowflake and s3. How would I then reference that dbt resource in the asset file(s)?
q

Qwame

06/12/2023, 6:35 PM
Copy code
@asset
def test_asset_with_dbt(dbt: DbtClientResource):
    dbt.run(....)
)
❤️ 1
o

owen

06/12/2023, 7:11 PM
hi @Benjamin Faught ! To expand a bit, you don't need access to a fully-formed DbtCliClientResource at the time that you're creating your
dbt_assets
. The assets that are created with
load_assets_from_dbt_project
will need a resource bound to them, but this can happen when you're creating your
Definitions
object. You can see an example project structure which incorporates dbt / a separate resources folder here: https://github.com/dagster-io/dagster/blob/master/examples/project_fully_featured/project_fully_featured/__init__.py
b

Benjamin Faught

06/12/2023, 8:11 PM
@owen Thanks! I have actually been using that project for reference. I guess the question is really, if I set a dbt resource like this in my resources folder:
dbt_resource = DbtCliClientResource(
project_dir=DBT_PROJECT_PATH, profiles_dir=DBT_PROFILES, ) can I use this resource as part of my dbt asset config in my assets folder?
dbt_assets = load_assets_from_dbt_project(
project_dir=DBT_PROJECT_PATH, profiles_dir=DBT_PROFILES, use_build_command=True, ) rather than specifying the project_dir and profiles_dir in both? Hopefully that makes sense. Again, still trying to get some sea legs with it.
@Qwame That's how to define a single asset from a dbt project as opposed to importing from project or manifest?
o

owen

06/12/2023, 8:21 PM
ah I see what you mean here -- the short answer is that you will need to specify those arguments in both places. the parameterization of the resource is used at runtime (when dbt models are actually getting executed), and the parameters of
load_assets_from_dbt_project
are used at definition time (to tell dagster what assets actually exist). If you were using
load_assets_from_dbt_manifest
, the project/profiles dir wouldn't even be specified in the first place (as dagster would not need to compile the dbt project to understand what's in it -- it'd just read directly from the manifest)
b

Benjamin Faught

06/12/2023, 8:25 PM
@owen Ahhhhhh that makes sense. Thanks for the help!
🌈 1